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ISO/IEC FCD 14496-JO 
Foreword 

The International Telecommunication Union (ITU) is the United Nations specialized agency in the field of 
telecommunications. The ITU Telecommunication Standardization Sector (ITU-T) is a permanent organ of ITU. ITU-T 
is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to- 
standardizing telecommunications on a worldwide has is. The World Telecommunication Standardization Assembly 
(WTSA), which meets every four years, establishes the topics for study by the ITU-T study groups which, in turn, 
produce Recommendations on these topics. The approval of ITU-T Recommendations is covered by the procedure laid 
down in WTSA Resolution I. In some areas of information technology which fall within ITU-T's purview, the necessary 
standards are prepared on a collaborative basis with ISO and IEC. 

ISO (the International Organisation for Standardisation) and IEC (the International Electrotechnical Commission) form 
the specialised system for world-wide standardisation. National Bodies that are members of ISO and IEC participate in 
the development of International Standards through technical committees established by the respective organisation to 
deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. 
Other international organisations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the 
work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTCI. 
Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. 
Publication as an International Standard requires approval by at least 75% of the national bodies casting a vote. 

This Recommendation | International Standard is being submitted for approval to the ITU-T and ISO/IEC JTC1/SC29. It 
was prepared jointly by ITU-T SG16 Q.6 also known as VCEG (Video Coding Experts Group) and by ISO/IEC 
JTCI/SC29/WG I I, also known as MPEG (Moving Picture Experts Group). VCEG was formed in 1997 to maintain prior 
ITU-T video coding standards and develop new video coding standard(s) appropriate for a wide range of conversational 
and non-conversational services. MPEG was formed in 1988 to establish standards for coding of moving pictures and 
associated audio for various applications such as digital storage media, distribution and communication. 

In this Recommendation | International Standard Annexes A through E contain normative requirements and are an 
integral part of this Recommendation | International Standard. 
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0 Introduction 
0.0 Prolog 

As processing power and memory costs have reduced, network support for coded video data has diversified, and 
advances in video coding technology have progressed, the need has arisen for an industy standard for compressed video 
representation with substantially increased coding efficiency and enhanced robustness to network environments. Toward 
these ends the ITU-T video coding experts group (VCEG) and the ISO/IEC moving picture experts group (MPEG) 
formed a joint video team (JVT) in 2001 for development of a new ITU-T Recommendation | International Standard. 

0.1 Purpose 

This Recommendation | International Standard was developed in response to the growing need for higher compression of 
moving pictures for various applications such as video conferencing, digital storage media, television broadcasting, 
internet streaming and communication. It is also designed to enable the use of the coded video representation in a 
flexible manner for a wide variety of network environments. The use of this Recommendation | Insternational Standard 
allows motion video to be manipulated as a form of computer data and to be stored on various storage media, transmitted 
and received over existing and future networks and distributed on existing and future broadcasting channels. 

0.2 Application 

This Recommendation | International Standard is designed to cover a broad range of applications for video content 
including but not limited to the following: 

CATV Cable TV on optical networks, copper, etc. 

DBS Direct broadcast satellite video services 

DSL Digital subscriber line video services 

DTTB Digital terrestrial television broadcasting 

ISM Interactive storage media (optical disks, etc.) 

MMM Multimedia mailing 

MSPN Multimedia services over packet networks 

RTC Real-time conversational services (videoconferencing, videophone, etc.) 

RVS Remote video surveillance 

SSM Serial storage media (digital VTR, etc.) 

0.3 Profiles and levels 

This Recommendation | International Standard is designed to be generic in the sense that it serves a wide range of 
applications, bit rates, resolutions, qualities and services. Applications should cover, among other tilings, digital storage 
media, television broadcasting and real-time communications. In the course of creating this Specification, various 
requirements from typical applications have been considered, necessary algorithmic elements have been developed, and 
they have been integrated into a single syntax. Hence, this Specification will facilitate video data interchange among 
different applications. 

Considering the practicality of implementing the full syntax of this Specification, however, a limited number of subsets 
of the syntax are also stipulated by means of "profile" and "level". These and other related terms are formally defined in 
clause 4. 

A "profile" is a subset of the entire bitstream syntax that is defined by this Recommendation | International Standard. 
Within the bounds imposed by the syntax of a given profile it is still possible to require a very large variation in the 
performance of encoders and decoders depending upon the values taken by parameters in the bitstream such as the 
specified size of the decoded pictures. It is currently neither practical nor economic to implement a decoder capable of 
dealing with all hypothetical uses of the syntax within a particular profile. 

In order to deal with this problem, "levels" are defined within each profile. A level is a defined set of constraints imposed 
on parameters in the bitstream. These constraints may be simple limits on numbers. Alternatively. they may take the form 
of constraints on arithmetic combinations of the parameters (e.g. frame width multiplied by frame height multiplied by 
frame rate). 



.1100641A_I_> 



ITU-T Rec. H.264 (2002 E) 



xiii 



ISO/IEC FCD 14496-10 



Coded video content conforming to this Specification uses a common syntax. In order to achieve a subset of the 
complete syntax, flags and parameters are included in the bitstream that signal the presence or otherwise of syntactic 
elements that occur later in the bitstream. In order to specify constraints on the syntax (and hence define a profile), it is 
thus only necessary to constrain the values of these flags and parameters that specify the presence of later syntactic 
elements. 



0,4 Overview of the syntax 

The coded representation defined in the syntax achieves a high compression capability while preserving image quality. 
The algorithm is not lossless as the exact sample values are not preserved through the encoding and decoding processes. 
Obtaining good ima§e quality at the bit rates of interest demands very high compression, which is not achievable with 
intra picture coding alone. The need for random access, however, is best satisfied with pure intra picture coding. The 
choice of the techniques is based on the need to balance a high image quality and compression capability with the 
requirement to allow random access into the coded video data stream. 

A number of techniques may be used to achieve high compression. The expected encoding algorithm (not specified in 
this Recommendation | International Standard) first uses block-based motion compensation to reduce temporal 
redundancy. Motion compensation is used both for causal prediction of a current picture from one or more previous 
pictures, and for non-causal prediction from future pictures in decoder output order. Motion vectors may be defined for a 
variety of region sizes in the picture. The prediction error is then further compressed using a transform to remove spatial 
correlation before it is quantised, producing an irreversible process that discards less important information while 
forming a close approximation to the source pictures. Finally, the motion vectors are combined with the quantised 
transform coefficient information and encoded using either variable length codes or arithmetic coding. 

0.4.1 Temporal processing 

Because of the conflicting requirements of random access and highly efficient compression, three main picture types are 
defined. Intra coded pictures (l-pictures) are coded without reference to other pictures. They provide access points to the 
coded sequence where decoding can begin, but are coded with only moderate compression. Inter-coded pictures CP- 
pictures) are coded more efficiently using motion compensated prediction of each block of sample values from some 
previously decoded picture selected by the encoder. Bi-prediclive pictures (B-pictures) provide the highest degree of 
compression but require a higher degree of memory access capability in the decoding process, as each block of sample 
values in a B picture may be predicted using a weighted average of two blocks of motion-compensated sample values. 

The organisation of the three picture types in a sequence is flexible, and the order of the decoding process is generally 
not the same as the order of the source picture capture process in the encoder or the output order from the decoder for 
display. The choice is left to the encoder and will depend on the requirements of the application. Figure Intro- 1 
illustrates one limited and example of the relationship among the three different picture types. Significantly different 
inter-picture dependency relationships are also allowed at the discretion of the encoder within limits specified by the 
profile and level. The decoding order is specified such that the decoding of pictures that use inter-picture prediction 
follows later in decoding order than other pictures that arc referenced in the decoding process. 
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Figure Intro. 1 - Example of temporal picture structure 
0.4.2 Coding interlaced video 

Each frame of interlaced video consists of two fields which are separated in capture time. This Recommendation I 
International Standard allows either the representation of complete frames or the representation of individual fields 
Frame encoding or field encoding can be adaptively selected on a picture-by-picture basis and also on a more localized 
basis within a coded frame. Frame encoding is typically preferred when the video scene contains significant detail with 
limited motion. Field encoding, in which the second field can be predicted from the first, works better when there is fast 
movement. 



0.4.3 



Macroblocks and motion segmentations 



As in previous video coding Recommendations and International Standard, a macroblock consisting of a 16x16 block of 
luma samples and a two corresponding blocks of chroma samples is used as the basic processing unit of the video 
decoding process. 

The selection of a motion compensation unit is a result of a trade-off between the coding gain provided by using motion 
information and the quantity of data needed to represent it. In this Recommendation | International Standard the motion 
compensation process can form segmentations for motion representation as small as 4x4 in size, using motion vector 
accuracy of one quarter or one-eighth of a sample grid spacing displacement. The inter prediction process for motion 
compensated prediction of a sample block can also involve the selection of the picture to be used as the reference picture 
from a number of stored previously-decoded pictures. 

In frame encoding, the prediction from the previous reference frame can itself be either frame-based or field-based, 
depending on the type of the motion vector information and other information that is encoded within the compressed 
picture representation. Motion vectors are encoded differentially with respect to predicted values formed from nearby 
encoded motion vectors. 

It is the responsibility of the encoder to calculate appropriate motion vectors or other data elements represented in the 
video data stream. This motion estimation process in the encoder and the selection of whether to use inter-picture 
prediction for the representation of each region of the video content is not specified in this Recommendation I 
International Standard. 



0.4.4 



Spatial redundancy reduction 



Both source pictures and prediction errors have high spatial redundancy. This Recommendation | International standard 
is based on the use of a block-based transform method for spatial redundancy removal. After motion compensated 
prediction or spatial-based prediction from previously-decoded samples within the current picture, the resulting 
prediction error is split into 4x4 blocks. These are converted into the transform domain where they are quantised. After 
quantisation many of the transform coefficients are zero or have low amplitude and can thus be represented with a small 
amount of encoded data. The processes of transformation and quantization in the encoder are not specified in this 
Recommendation | International Standard. 
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Information technology Coding of audio-visual objects ~ 
Part 10: 

Advanced video coding 

1 Scope 

This document specifies ITU-T Recommendation H.264 | ISO/IEC International Standard ISO/IEC 14496-10 video 
coding. 

2 Normative references 

The following Recommendations and International Standards contain provisions which, through reference in this text 
constitute provisions of this Recommendation | International Standard. At the time of publication, the editions indicated 
were valid. All Recommendations and Standards are subject to revision, and parties to agreements based on this 
Recommendation | International Standard are encouraged to investigate the possibility of applying the most recent 
edition of the Recommendations and Standards listed below. Members of IEC and ISO maintain registers of currently 
valid International Standards. The Telecommunication Standardization Bureau of the ITU maintains a list of currently 
valid ITU-T Recommendations. 

- ITU-T Recommendation T.35 (2000), Procedure for the allocation of ITU-T defined codes for non- 
standard facilities 

3 Definitions 

For the purposes of this Recommendation | International Standard, the following definitions apply. 

3.1 AC coefficient: Any transform coefficient for which the frequency index in one or both dimensions is non- 
zero. 

3.2 B slice: A bi-predictive slice; A slice that is coded in a manner in which a weighted average of two inter 
prediction blocks may be used for inter prediction. 

3.3 bitstream: A sequence of bits that forms the representation of data and coded fields and frames. 

3.4 block: An N-column by M-row array of samples, or NxM array of transform coefficients. 

3.5 bottom field: One of two fields that comprise a frame. Each row of a bottom field is spatially located 
immediately below a corresponding row of a top field. 

3.6 byte: A sequence of 8 bits, ordered from the first and most significant bit on the left to the last and least 
significant bit on the right. 

3.7 byte aligned: A bit in a bitstream is byte-aligned if its position is a multiple of 8 bits from the first bit in the 
bitstream. 

3.8 byte stream format: A NAL unit stream containing start code prefixes and NAL units as per Annex B. 

3.9 category: For slice layer and lower layer syntax elements, specifies the allocation of syntax elements to data 
structures for data partitioning. It may also be used by the systems layer to refer to classes of syntax elements in a 
manner not specified in this Recommendation | International Standard. 

3.10 chroma: An adjective specifying that a sample array or single sample is representing one of the two colour 
difference signals related to the primary colours. The symbols used for the chroma array or sample are Cr and Cb. 

3.11 coded field: A coded representation of afield. 

3.12 coded frame: A coded representation of a frame. 

3.13 coded pictures input buffer: A first-in first-out (FIFO) buffer containing coded pictures in decoding order 
specified in the video buffering verifier in Annex C. 

3.14 coded representation: A data element as represented in its coded form. 
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3.15 comm n intermediate format (OF): A video frame that is 22 rnacroblocks wide and 18 rnacroblocks high. 

3.16 component: An array or single sample from one of the ihrec arrays (luma and two chroma) that make up a 
field or frame. 

3.17 context: The numerical value of the context variable when decoding a symbol. 

3.18 context modelling: The choice and specification of prior decoded symbols that are to be used in the decoding 
of a symbol. 

3.19 context variable: Specified for each symbol by an equation containing the recently decoded symbols as 
defined by context modelling. 

3.20 dangling field: A field for which there is no adjacent field carrying the same frame number. 

3.21 decoding order: The order in which the coded pictures are to be decoded. 

3.22 data partitioning: A method of partitioning selected syntax elements into syntactical structures based on a 
categorization of the syntax elements. 

3.23 DC coefficient: The transform coefficient for which the frequency index is zero in both dimensions. 

3.24 decoded picture: A decoded picture is obtained by decoding a coded picture. A decoded picture is cither a 
decoded frame, or a decoded field. A decoded field is a decoded top field or a decoded bottom field. 

3.25 decoded pictures buffer: A buffer specified in the video buffering verifier in subclause C.l. The decoded 
picture buffer comprises the reference picture buffer and the picture reordering buffer. 

3.26 decoder: An embodiment of a decoding process. 

3.27 decoding process: The process specified in this Recommendation | International Standard that reads a NAL 
unit stream and produces decoded fields or frames. 

3.28 direct prediction: An inter prediction for a block lor which no motion vector is decoded. 

3.29 encoder: An embodiment of an encoding process. 

3.30 emulation prevention byte: A byte having a fixed value present within a NAL unit. The presence of 
emulation prevention bytes ensures that no sequence of consecutive byte-aligned bytes in the NAL unit contains a start 
code prefix. 

3.31 encoding process: A process, not specified in this Recommendation | International Standard, that reads a 
sequence of fields and frames and produces a conforming NAL unit stream as specified in this Recommendation | 
International Standard. 

3.32 field: An assembly of alternate rows of a frame. A frame is composed of two fields, a top field and a bottom 
field. 

3.33 flag: A variable which can take one of only two possible values. 

334 frame: A frame contains sampled and quantized luma and chroma data of all rows of a of a video signal frame. 
A frame consists of two fields, a top field and a bottom field. For interlaced video signal, one of these fields is sampled 
temporally later than the other. 

3.35 intra prediction: A prediction derived from the decoded samples of the same decoded picture. 

3.36 instantaneous decoder refresh (IDR) picture: A special / picture that causes the decoder to mark all 
reference pictures in the decoded pictures buffer as un-used immediately before decoding the IDR picture, and to 
indicate that later coded pictures can be decoded without inter prediction from any picture decoded prior to the IDR 
picture. 

3.37 inter coding: Coding of a block, macroblocK slice, or picture that uses information from both, within the 
picture and from other pictures. 

3.38 inter prediction: A prediction derived from decoded samples of pictures other than the current decoded 
picture. Inter prediction is a collective term for the prediction process in P, SP, and B rnacroblocks. 

3.39 intra coding: Coding of a block, macroblock, slice or picture that uses intra prediction. 

3.40 I picture: An intra picture; A picture that is coded using prediction only from decoded samples within the 
same picture. 

3.41 inverse transform: A part of the decoding process by which a block of scaled transform coefficient levels is 
converted into a block of spatial-domain samples. 
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3.42 layer: One of a set of syntaciicaJ structures in a non-branching hierarchical relationship. Higher layers contain 
lower layers. The coding layers are the picture, slice, reference picture selection, macroblock, 8x8 block ana 4x4 block 

3.43 level: A defined set of constraints on the values which may be taken by the parameters of this 
Recommendation | International Standard. The same set of level definitions are used with all profiles, but individual 
implementations may support a different level for each supported profile. In a different context, level is the value of a 
transform coefficient prior to scaling. a 

3 44 long SCP: A start code prefix which is used in the construction of the byte stream format for NAL unit 

TcpT^I 1 ma " dat0r y J f Sl3r ! ° f 3 ^ded picture in byte stream format. Optionally i, can be used instead of a short 
at the start of a coded slice or Jower coding layers. 

if a L,J„Tnr A " ad Jf live ^ if y in 8 Aat i . sample array or single sample is representing the monochrome signal 
related to the primary colours. The symbol used for luma is Y. 

3.46 macroblock: The 1 6x 1 6 luma samples and the two corresponding blocks of chroma samples. 
l^crobloT^Xt:^ ^ ^ SCQn ^ nUmber ° f 3 maCr ° blOCk With Zero for < he ">P 

3 ' 48 hi " ,aC ™ bl0ck a,,oca tio" "nap: A means of partitioning the macroblocks of a picture into slice groups The 

'sx^ssss: an array of numbere one for each coded maM - — sr 0uP to wf.chTJ: 

t 49 , , . macrob '« ck t 1 of * ion: . The two dimensional coordinates of a macroblock in a picture designated by (x y) For 
the top left macroblock of the picture (x,y)=(0,0). x is incremented by 1 for each macroblock column from left to righf v 
is incremented by I tor each macroblock row from top to bottom. g y 

3.50 macroblock pair: A pair of vertically-contiguous macroblocks in a picture that is coupled for use in 
macroblock-adaptive frame/field decoder processing. coupled tor use in 

3.51 Mbit: 1 000 000 bits. 

m tion compensation: Part of the inter prediction process for sample values, using previously decoded samples that are 
spatially displaced as signalled by means of motion vectors. mmpies uiai are 

3.53 motion vector: A two-dimensional vector used for motion compensation that provides an offset from the 
coordinate position in the decoded picture to the coordinates in a reference picture. 

3 54 NAL unit: A syntax structure containing an indication of the type of data to follow and bytes containing that 
data interspersed as necessary with emulation prevention bytes. 8 at 

cement. Stream: A SCquence ° f NAL units con,aini ng the syntax structures associated with the coded video 

Lmin, a nH e ! WOrk ^ (NAL): A definition of syntax structures and additional information including 

framing and timing that are supported by a system Wl " s 

3.57 non-reference picture: a decoded picture that is marked as not used for inter prediction. 

3.58 opposite parity: The opposite parity of top is bottom, and vice versa. 

3.59 output order: The order in which the decoded pictures are intended for output. 

o^rnf n^ tPUt re0 f? e ™ g delay: A J!*'"* be,ween decodi "g * coded picture and its output that is caused when the 
order of pictures specified for output is different from the order specified for decoding. 

3.61 parity: The parity of a field can be top or bottom. 

3.62 partitioning: The division of a set into sub-sets such that each element of the set is in exactly one of the sub- 

3.63 P slice: A predictive slice; A slice that is coded using inter prediction from previously-decoded reference 
pictures, using at most one motion vector and reference picture index to predict the sample values of each S 

3.64 picture: A collective term for a field or a frame. 

3.65 picture order count: Picture position in output order relative to the latest IDR picture in decoding order. 
frot the o^tordTr^^ ^ * re '° rderine *° * C ° ded wheD the ° rder is 
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3.67 prediction: An embodiment of the prediction process, 

3.68 prediction process: The use of a predictor to provide an estimate of the sample value or data element currently 
being decoded. 

3.69 prediction residual: The difference between the value of a source sample or data element and its predictor. 

3.70 predictor: A combination of previously decoded sample values or data elements used in the decoding process 
of subsequent sample values or data elements. 

3.71 probability model: The set of probability distributions used by the arithmetic decoding process when decoding 
a symbol. The context determines which probability distribution is to be used when decoding a particular symbol at a 
particular point, block, macroblock, etc. in the picture. For each symbol, the number of probability distributions in the 
set is equal to the number of possible values for the context variable, i.e., the number of contexts. 

3.72 profile: A specified subset of the syntax of this Recommendation | International Standard. 

3.73 quantisation parameter: A parameter used by the decoding process for scaling of transform coefficient levels. 

3.74 quarter common intermediate format (QCIF): A video frame thai is 11 macroblocks wide and 9 
macroblocks high. 

3.75 random access: The ability to start the decoding of a coded NAL unit stream at a point other than the 
beginning of the stream and recover and exact or approximate representation of the decoded pictures represented by that 
NAL unit stream. 

3.76 raster scan: A mapping of a rectangular two-dimensional pattern to a one-dimensional pattern such that the 
first entries in the one-dimensional pattern arc from the first row of the two-dimensional pattern scanned from left to 
right, followed similarly by the second, third, etc. rows of the pattern each scanned from left to right. 

3.77 reference field: A reference field is used for inter prediction when P macroblocks, SP macroblocks, and B 
macroblocks of a coded field or a coded frame are decoded. 

3.78 reference frame: A reference frame is used for inter prediction when P macroblocks, SP macroblocks, and B 
macroblocks of a coded frame are decoded. 

3.79 reference index list: A list of indices that is assigned to the reference pictures in the reference picture buffer. 

3.80 reference index list 0: The list of reference indices for use in reference list 0 prediction for a P, B, or SP slice. 
All inter prediction used for P and SP slices is considered reference list 0 prediction. The reference index list 0 is one of 
two reference index lists used for a B slice, with the other being the reference index list 1 . 

3.81 reference list 0 motion vector: A motion vector associated with a reference index pointing into the reference 
index list 0. 

3.82 reference list 0 prediction: Inter prediction of the content of a slice using a reference index into the reference 
index list 0. 

3.83 reference index list 1: A list of reference indices defined for use in inter prediction for a B slice. The 
reference index list! is one of two lists of reference itulices used by a B slice, with the other being the reference index list 
0. 

3.84 reference list 1 motion vector: A motion vector associated with a reference index pointing into the reference 
index list I . 

3.85 reference list 1 prediction: Inter prediction of the content of a B slice using a reference index into the 
reference index list I. 

3.86 reference picture: A picture containing samples that are used for inter prediction. 

3.87 reference picture buffer: A (part of the decoded pictures ?) buffer containing the reference pictures. 

3.88 reference picture buffer management: specifies in the coded data, how the decoding process applies to the 
decoded pictures buffer and in particular to the reference picture buffer. 

3.89 reserved: The term "reserved", when used in the clauses defining some values of a particular syntax element 
means that these values may be used in extensions of this Recommendation | International Standard by ITU-T | ISO/IEC, 
and that these values shall not be used unless so specified. 

3.90 residual: The decoded difference between a prediction of a sample or data element and its decoded value. 
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3.91 run: A number of consecutive data elements represented in the decoding process. In one context, the number 
of zero-valued transform coefficients preceding a non-zero transform coefficient, in the block scan order. In another 
context, the number of skipped macroblocks. 

3.92 sample aspect ratio (SAR): Specifies the distance between luma samples. It is defined as the vertical 
displacement of the rows of luma samples in a frame divided by the horizontal displacement of the luma samples Thus 
its units are (metres per row) + (metres per sample). 

3.93 scaling: The process of scaling the transform coefficient levels resulting in transform coefficients. 

3.94 short SCP: A start code prefix which is used in the construction of a byte stream format of coded data. It can 
be used instead of a long SCP at the start of a coded slice or lower coding layers. 

3.95 skipped macroblock: A tnacroblock for which no data is coded other than an indication that the macroblock is 
to be decoded as "skipped". This indication may be common to several macroblocks. 

3.96 slice: An integer number of macroblocks ordered contiguously in raster scan order within a particular slice 
group. Although a slice contains macroblocks that are contiguous in raster scan order within a slice croup these 
macroblocks are not necessarily contiguous within the picture. The addresses of the macroblocks are derived from the 
address of the first macroblock and the slice group parameters. 

3.97 slice group: A sub-set of the macroblocks of a picture. The division of the picture into slice <roups is a 
partitioning ot the picture. The partition is specified by the slice group parameters. 

3.98 slice header: A part of a coded slice containing the coded representation of data elements pertaining to the 
slice data that follow the slice header. F 5 

3.99 SI picture: A switching 1 picture; A picture that is coded using prediction only from decoded samples within 
the same picture, encoded such that it can be reconstructed identically to another SP slice or SI slice as soecified in 
clause II. * 

3.100 SI slice: A switching I slice; A slice that is coded using prediction only from decoded samples within the same 
slice, encoded such that it can be reconstructed identically to another SP slice or SI slice, as specified in subclause 1 1. 

3.101 source (input): Term used to describe the video material or some of its attributes before encoding. 

3.102 SP slice: A switching P slice; A slice that is coded using inter prediction from previously-decoded reference 
pictures, using at most one motion vector and reference picture index to predict the sample values of each block encoded 
such that it can be reconstructed identically to another SP slice or SI slice, as specified in subclause 1 1 . 

3 103 start code prefix (SCP): One of a set of unique codes embedded in the byte-stream format that are used for 
identifying the beginning of a coding layer. Emulation of start code prefixes is prohibited within NAL units. 

3.104 string of data bits (SODB): An ordered sequence of some finite number of bits, in which the left-most bit is 
considered to be the first and most significant bit (MSB) and the right-most bit is considered to be the last and least 
significant bit (LSB). 

3.105 symbol: A syntax element, or part thereof, to be decoded. 

3.106 top field: One of iwo fields that comprise a frame. Each row of a top field is spatially located immediately 
above the corresponding row of the bottom field. 

3.107 transform coefficient: A scalar considered to be in a frequency domain that is associated with a particular 
two-dimensional frequency index in the inverse transform of the decoding process. 

3.108 variable length coding (VLC): A reversible procedure for entropy coding that assigns shorter code-words to 
frequent symbols and longer code-words to less frequent symbols. 

3.109 video buffering verifier (VBV): A hypothetical decoder that is connected to the output of the encoder Its 
produce 15 l ° Pr ° V,de 3 C ° nSlrainl ° n the variabilit y of the N AL unit stream that an encoder or editing process may 

3#1 i? , Pr ° fi,e decoder: A decoder able to decode coded data conforming to the specifications of the XYZ 

profile (with XYZ being any of the defined Profile names). 

3.111 zig-zag scan: A specific sequential ordering of transform coefficients from (approximately) the lowest spatial 
frequency to the highest. J/ F 

4 Abbreviations 

4.1 ABT: Adaptive Block size Transform 
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4.2 


CABAC: Context-based Adaptive Binary Arithmetic Coding 


4.3 


CAVLC: Context-based Adaptive Variable Length Coding 


4.4 


CIF: Common Intermediate Format 


4.5 


DP A: Data Partition type A 


4.6 


DPB: Data Partition type B 


4.7 


DPC: Data Partition type C 


4.8 


FCC: Federal Communications Commission 


4.9 


FIFO: First-In, First-Out 


4.10 


IDR: Instantaneous Decoder Refresh 


4.11 


LPS: Least Probable Symbol 


4.12 


LSB: Least Significant Bit 


4.13 


MB: Macroblock 


4.14 


MPS: Most Probable Symbol 


4.15 


MSB: Most Significant Bit 


4.16 


NAL: Network Abstraction Layer 


4.17 


QCIF: Quarter Common Intermediate Format 


4.18 


RBSP: Raw Byte Sequence Pay load 


4.19 


SAR: Sample Aspect Ratio 


4.20 


SCP: Start Code Prefix 


4.21 


SEI: Supplemental Enhancement Information 


4.22 


SMPTE: Society of Motion Picture and Television Engineers 


4.23 


SODB: String Of Data Bits 


4.24 


VCL: Video Coding Layer 


4.25 


VBV: Video Buffering Verifier 


4.26 


VLC: Variable Length Coding 


5 


Conventions 



The mathematical operators used to describe this Specification arc similar to those used in the C programming language. 
However, integer divisions with truncation and rounding arc specifically defined. Numbering and counting loops 
generally begin from zero. 

5.1 Arithmetic operators 

The following mathematical and logical operators arc defined as follows 
+ Addition 

Subtraction (as a binary operator) or negation (as a unary operator) 
++ Increment, i.e. x++ is equivalent to x = x + I 

— Decrement, i.e. x — is equivalent to x = x - I 

* J Multiplication 
A Power 

/ Integer division with truncation of the result toward zero. For example, 7/4 and -7/-4 are truncated to 

I and -7/4 and 7/-4 are truncated to -1 . 
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DIV Integer division with truncation of the result toward minus infinity. For example 3 DIV 2 is rounded 
to I , and -3 DIV 2 is rounded to -2. u 

Used to denote division in mathematical equations where no truncation or rounding is intended 

b 

2/(0 The summation of the/(i) with i taking all integer values from a up to and including b. 
a % b Modulus operator. Remainder of a divided by b, defined only for a and b both posit,ve integers 

5.2 Logical operators 

a && b Boolean logical "and" of a and b 
a 1 1 b Boolean logical "or" of a and b 
! Logical NOT 

5.3 Relational operators 

> Greater than 



>= 



Greater than or equal to 



< Less than 



<= 



Less than or equal to 
Equal to 
Not equal to 



5.4 Bit-wise operators 

& AND 
I OR 



a » b Arithmetic right shift of ■ two's complement integer representation of a by b binary digits This 

Ihift 'h n i. ,S h ,° y f0 ^ si ' ive va,ues of *• Bi " s^ted ^to the MSBs as a result of the right 

shift shall have a value equal to the MSB of a prior to the shift operation. g 

Q K< b ^t meliC J e ? S 5 ift .° f / tW °' S com P ,ement representation of a by b binary digits This 

function is defined only for positive values of b. * ms 



5.5 Assignment 

= Assignment operator 

5.6 Functions 



Sign(^)=j 1 ' 

l-l ; ;t<0 (5-0 



Abs(^)= ' x x ~° 



I 



-x ; x<0 (5-2) 



\a ; c<a 
CIip3( a, b, c) = \b ; ob 



(5-3) 
otherwise 



Clip l(*) = Clip3(0, 255, x) - 5 
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Ceil( x ) rounds x up to the nearest integer. Defined only for non-negative values of x. (5-5) 
Log2( x ) returns the base-2 logarithm of x. (5~ 6 ) 



6 Source coder 

6.1 Picture formats 

The imaoc width and height of the decoded luma memory arrays are multiples of 16 samples. Decoder output picture 
sizes that are not a multiple of 16 in width or height can be specified using a cropping rectangle. This Recommendation | 
International Standard represents colour sequences using 4:2:0 chroma sampling. 

The nominal vertical and horizontal locations of luma and chroma samples in frames are shown in Figure 6-1. 
Alternative chroma sample locations may be indicated in video usability information syntax (see Annex E). 
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Guide: 











X = Location of luminance sample 
O = Location of chrominance sample 

Figure 6-1 - Nominal vertical and horizontal locations of 4:2:0 luma and chroma samples in a frame 



This Recommendation | International Standard describes decoding of video that contains either progressive-scan or 
intcrlaced-scan frames, which may be mixed together in the same sequence. 

A decoded frame of video contains two fields, the top field and the bottom field, which arc interleaved. The first (i.e 
top) third, fifth, etc. rows of a decoded frame are the top field rows. The second, fourth, sixth, etc. rows of a decoded 
frame arc the bottom field rows. A top field picture consists of only the top field rows of a decoded frame. A bottom field 
picture consists of only the bottom field rows of a decoded frame. 

The two decoded fields of an interlaced frame are separated in time. They may be decoded separately as two fields or 
together as a frame. 

NOTE - A progressive frame should always be coded as a single frame picture. However, a progressive frame is still considered to 

consist of two fields (at the same instant in time). 
The nominal vertical and horizontal locations of luma and chroma samples in interlaced frames are shown in Figure 6-2. 
The vertical sampling locations of the chroma samples in a top field of an interlaced frame are specified as shifted up by 
1/4 luma sample heFght relative to the field-sampling grid in order for these samples to align vertically to the usual 
location relative to the full-frame sampling grid. The vertical sampling locations of the chroma samples in a bottom field 
of an interlaced frame are specified as shifted down by 1/4 luma sample height relative to the field-sampling grid in order 
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for these samples to align vertically to the usual location relative to the full-frame sampling grid The horizons 
samphng iocac.ons of the chroma samples are specified as unaffected by the application of interlaced field cooing 



Top Bottom Top 

Field Field Field 

x x 

o o 



X 



o 

X 



X X 
o O 
X 



o 

X 



X 



Time 

X = Luminance Sample 
O = Chrominance Sample 



Figure 6-2 - Nominal vertical and temporal sampling locations of samples in 4:2:0 interlaced frames 



6.2 Spatial subdivision of a picture into macroblocks 

Pictures are divided into macroblocks. For instance, a QCIF picture is divided into 99 macroblocks as indicated in Figure 



Figure 6-3 - A picture with 11 by 9 macroblocks (QCIF picture) 



6.3 Calculation of the macroblock address 

s^**^ " PiCtUre Parameter U °' ^ maCr ° b,OCk addrCSS Ca,CUlatio " » -ursive.y 

L?oM dC J S !i Ce C ° mainS hS S ' ice hCader the macr °°»°ck address of the first macroblock in the coded slice The 
macroblock allocat.on map conveys «he slice group identifier of the first macroblock in a slice. 



DOCID: <XP 1 10O64IA__l_> 



ITU-T Rec. H.264 (2002 E) 



9 



ISO/IEC FCD 14496-10 

2 Let g be the slice group identifier of the most recently decoded macroblock of a given coded slice The next 
macroblock address is found by searching the macroblock allocation map in scan order for the next macroblock that has 
the same slice group identifier g. 

When mb_adaptive_frame_field_f1ag in the picture parameter set is I, the macroblock pair address calculation is 
recursively specified as follows (sec Figure 6-4): 

1 A coded slice contains in its slice header the macroblock pair address of the first macroblock pair in the coded slice. 
The slice group index of the first macroblock pair in a coded slice may be found by referencing the macroblock 
allocation map. 

2 Let " be the slice "roup identifier of the most recently decoded macroblock pair of a given coded slice. The next 
macroblock pair address is found by searching the macroblock allocation map in scan order for the next macroblock pair 
that has the same slice group identifier g. 

NOTE - This note describes one of many possible implementations of the macroblock address calculation (the macroblock pair 
address could be calculated with a similar algorithm). . . 

Assume the availability of a one-dimensional array m with as many entries as there are macroblocks or macroblock pairs in the 
ni'cturo and that is initialized with a one-dimensional representation of the macroblock allocation map ol the picture parameter set 
(in on^'of " he several explicit or implicit forms defined there). Let n be the macroblock address of the last decoded macroblock of 
a given slice. The next macroblock address is calculated by the following three steps: 

1. Identify the slice group of the macroblock n by using n as an index into m. 

2. Search in m in ascending order, starling with n, for the next entry that has the same slice group identifier. 

3. This is the macroblock address for the next macroblock or macroblock pair of the coded slice. 

NOTE - a coded slice may consist of one slice NAL unit or. when data partitioning is used, of three NAL units OPA. OPB. and DPC. 
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An MB pair 



Figure 6-4 - Partitioning of the decoded frame into macroblock pairs. An MB pair can be coded as two frame 
MBs, or one top-field MB and one bottom-field MB. The numbers indicate the scanning order of coded MBs. 



6.4 Assignment of symbols within a macroblock 

Figures 6-5 indicates how a macroblock or sub macroblock is partitioned with each luma block and associated chroma 
blocks being motion-compensated using a separate motion vector and (for luma blocks larger or equal to 8x8 samples 
and associated chroma blocks) using a separate reference picture index. If the ABT feature is used, the transform for 
residual coding is adapted to the partitioning pattern as well. 
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16*8 



8*16 



8*8 



Macroblock 
types 





Sub macroblock 
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4*4 
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Figure 6-5 - Numbering of the vectors for the different blocks in raster scan order depending on the inter mode. 
For each block the horizontal component comes first followed by the vertical component. 
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Figure 6-6 shows the order of the assignments of syntax elements resulting from coding a macroblock to sub-blocks of 
the macroblock if the ABT feature is not used. The assignment order if the ABT feature is used is specified in 
Figure 12-1. 
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CBPY 8*8 block order 
(raster scan order in MB) 
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Luma 4x4 block order for 4x4 
intra prediction and 4x4 
residual coding 
(raster scan order within 8x8 
region nested in raster scan 
order of 8x8 regions) 



Luma 4x4 DC for Intra 1 6x1 6 
macroblock type 



Cb 


16 




18 




20 


M 
21 



Cr 



2x2 DC 



22 


W 


m 
24 


m 
25 



AC 



Chroma 4x4 block order for 
4x4 residual coding, shown 
as 16-25, and intra 4x4 
prediction, shown as 18-21 
and 22-25 (raster scan order 
in each 8x8 chroma region) 



Figure 6-6 - Ordering of blocks for codcd_block_pattern Y, 4x4 intra prediction, and 4x4 residual coding 



7 Syntax and semantics 

7 A Method of describing the syntax in tabular form 

The syntax is described in a manner that closely follows the C-languagc syntactic constructs. Syntax elements in the 
bitstrcam arc represented in bold type. Each syntax element is described by its name, its syntax category and descriptor 
for its method of coded representation. A decoder behaves according to the value of the syntax clement and on the values 
of previously decoded syntax elements. 

The syntax tables describe a superset of the syntax of all correct and error-free input bitstreams. Additional constraints on 
the syntax form may be specified in other clauses. An actual decoder must implement correct means for identifying 
entry points into the bitstream for proper decoding and to identify and handle errors in the bitstrcam. The methods for 
identifying and handling errors and other such situations are not described here. 

Following C-language conventions, a value of 'O 4 represents a FALSE condition in a test statement. The value TRUE is 
represented by ' I \ but any other value different than zero is also understood as TRUE. 
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The following table lists examples of pseudo code used to describe the syntax. When syntax element ann^r* 
.ndicates that a data element is read (extracted) from the bitstream and the bitstream pointer advances to Z bi KX" 
the last bit of the data element extracted. e ou TO,1 owmg 





Category 


| Descript r j 


/* A statement can be a syntax element with an associated syntax 

category ana aesenptor or can be an expression used to specify 

conditions for the existence, type, and quantity of svntax pipmpntc ^ ;« 

, vr v ' ^ vj«ja»niLjr kj t oyni<tA ciemenis, as in 

the following two examples */ 






syntax_element 


3 


1 e(v ) ! 


conditioning statement 










T 


/* A group of statements enclosed in curly brackets is a compound 
statement and is treated functionally as a single statement */ 











j — 


statement 






statement 




4 








} — 












/* A while structure indicates a test of whether a condition is true, and 
it true, indicates evaluation of a statement (or compound statement) 
repeatedly until the condition is no longer true */ 




— 


while( condition ) ~ ~ " 




I— 


statement ~ 




1 J 








1 * A do ... while ^trilPllirf 1 inrli/~?it*»c p»/ilno»irt^ ^.f ^ 

i w ... nimg auutiuic uiuicdies evaluation or a statement once 
followed by a test of whether a condition is true, and if true, indicates 
repeated evaluation of the statement until the condition is no longer true 






do 










j 


u/h i \f*( rr\r\s\ t f i /-\n % ' 

wniic^ cui luiuun i 




] 








/ /\n ir...eise structure indicates a test of whether a condition is 
true, and if the condition is true, indicates evaluation nf » nrimoru 
statement, otherwise indicates evaluation of an alternative statement 
The "else" part of the structure and the associated alternative statement 
is omitted if no alternative statement evaluation is needed */ 






if( condition ) 




— 


primary statement 






else ~ ~ ~~ — - 






alternative statement 













/* A for structure indicates evaluation of an initial statement, followed 
by a test of a condition, and if the condition is true, indicates repeated 
evaluation of a primary statement followed by a subsequent statement 
until the condition is no longer true */ 






for( initial statement; condition; subsequent statement ) 






primary statement 
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7.2 Definitions of functions and descriptors 

The functions presented here are used in the syntactical description. These functions assume the existence of a bitstream 
pointer with an indication of the position of the next bit to be read by the decoder from the bitstream. 

byte_aligned( ) 

. Returns TRUE if ihc current position in the bitstream is on a byte boundary, i.e., the next bit in the bitstream is 
the first bit in a byte. Otherwise it returns FALSE 

first_non_skip_mb_in_pair( ) 

. Returns TRUE if the current macroblock is the first macroblock in a macroblock pair or if the previous 
macroblock in the macroblock pair was skipped. Used only in maeroblock-adaptive frame/held coding. 

ncxt_bits( n ) 

. Provides the next bits in the bitstream for comparison purposes, without advancing the bitstream pointer. 
Provides a look at (he next n bits in the bitstream with n being its argument. If used wuhin an RBSP syntax 
structure or in a structure within an RBSP syntax structure, returns a non-matching value il lewer than n bits 
remain within the RBSP prior to the rbsp_trailing_biLs( ). If used within the bylc stream formal syntax specked 
in Annex B, returns a non-matching value if fewer than n bits remain within the byte sircam. 

more_rbsp_data( ) 

. Returns TRUE if there is more data in an RBSP before rbsp_trailing_bits( ). Otherwise it returns FALSE. The 
method for enabling determination of whether there is more data in the slices is speeded by the system (or in 
Annex B for systems that use the byte stream formal). 

total_cocff( ) - 

• Returns the number of coefficients from coeff_token. See subclause 7.3.5.3. 1 

trail ing_oncs( ) 

• Returns the trailing ones from eoeff_token. See subclause 7.3.5.3.1. 
slice_type( ) 

• Returns the coding type of slice. 

The following descriptors are used to describe the type of each syntax clement. 

• b(8): byte having any value ( 8 bits ). 

• uc(v): unsigned integer Exp-Golomb-coded syntax element with the left bit first. 

• se(v): signed integer Exp-Golomb-coded syntax element with the left bit first. 

• mc(v): mapped Exp-Golomb-coded syntax element with the left bit first. 

• cc(v): context-adaptive variable-length entropy-coded syntax element with the left bit first. 

• f(n): fixed-value bit string using n bits written (from left to right) with the left bit first. 

• i(n)* signed integer using n bits for a two's complement representation with most significant bit written first. If 
n is M v'\ the number of bits varies in a manner dependent on the value of other decoded data. 

. u(n): unsigned integer using a bits with most significant bit written first. If n is "v\ the number of bits varies in 
a manner dependent on the value of other decoded data. 

• xe(v)- extended Exp-Golomb-coded syntax element with left bit first. If indicating a selection from a list having 
only two alternatives, shall be interpreted as u(l). If indicating a selection from a list having more than two 
alternatives, shall be interpreted as uc(v). 

• ae{v)* context-adaptive arithmetic entropy-coded syntax element. Some syntax elements arc coded using 
CABAC when entropy_coding_mode - = 1 . This is indicated by specifying an alternative descriptor separated 
by a bar. 
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7.3 Syntax in tabular form 
7.3.1 NAL unit syntax 



naJ_unit( NumBytesInNALunit ) { 




— ; j 

Descriptor 


forbidden_bit 






nal_storage_idc 






nal_unit_type 




up; 


NumBytesInRBSP = 0 






for( i = 0; j < NumBytesInNALunit-] ; i++ ) { 






if( next_bits( 16 ) = = 0x0003 ) { 






rbsp[ NumBytesInRBSP++ ] 




b(8) 


i-m- 






emulation_prevention_by te /* = = 0x03 */ 




f(8) 


} else 






rbsp[ NumByteslnRBSP++ ] 




b(8) 


} 






J 







7.3.2 Raw byte sequence payloads and RBSP trailing bits syntax 

7.3.2.1 Sequence parameter set RBSP syntax 



seq_parameter_set_rbsp( ) { 


Category 


Oescrintor 


profile_idc 


0 


ue(v) 


level_idc 


0 


ue(v) 


seq_parameter_set_id 


0 


ue(v) 


log2_max_frame_num_minus4 


0 


ue(v) 


pic_order_cnt_type 


0 


ue(v) 


if( pic_order_cnt_type = = 0 ) 






log2_max_pic_order_cnt_minus4 


0 


ue(v) 


else if( pic_order_cnt_type = = 1 ) { 






offset_for_non_stored_pic 


0 


se(v) 


num_stored_frames_in_pic_order_cnt__cycle 


0 


ue(v) 


for( i = 0; i < num_stored_frames_in_pic_order_cnt_cycle; i++ ) 






ofTsetJbr_stored_frame 


0 


se(v) 


} 






nu m_of_ref_f ra mes 


0 


ue(v) 


required_frame_num_update_behaviour 


0 


u(l) 


pic_width_in_mbs_minusl 


0 


ue(v) 


pic_height_in_mbs_mimjsl 


0 


ue(v) 


send_filter_parameters_flag 


0 


u(D 


constrained_intra_pred_flag 


0 


u(l) 


mb_f rame_field_adaptive_flag 


0 


u(l) 


vui_seq_parameters_flag 


0 


u(l) 


if( vui_seq_parameters_fiag ) 






vui_seq_parameters( ) 


0 




rbsp_trailing_bits( ) 


0 




} 
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73.2.2 Picture parameter set RBSP syntax 



pic_parameter_set_rbsp( ) ( 


v-aiegory 


L/CoV. I l |J LvJ I 


pic_parameter_sct_id 


i 
i 


ue(v) 


seq_para mete r_se t_id 




uc \ v 7 


entropy_coding_mode 


i 
i 




motion_resoIution 


i 
l 


ue(v) 


adaptivc_block_size_transform_flag 


i 
i 




num_sIicc_groups_minusl 


1 
i 


ue(v) 


if( num_slicc _groups_minusl > 0 ) { 




— 


mb_allocation_map_type 


i 
i 


ue(v) 


if( mb„allocation_map_typc = = 0 ) 






for( i = 0; i <= num_slicc_groups_minusl ; i++ ) 






runjength 


i 
1 


ue(v) 


else if( mb_allocation_map_iype = = 2 ) 






for( i = 0; i < num_mbs_in_pic; i++ ) 






slicc_group_id 


i 


u(v; 


clscif( mb_allocation_map_typc == 3 ) 






for( i = 0; i < nuni slice_<z;roups_minusI ; i++ ) { 






ti\r% Ipff mh 


1 


u(v) 


Krkttnm rifht mh 


1 


u(v) 


J 






iff mh allocation man type = = 4 1 1 
mb__allocation_map_typc = = 5 | | 
mb allocation map lypc = = 6) { 






slice_group_change_direction 


1 


u( I ) 


slice_group_changc_rate_minus 1 


i 


ue(v) 


} 






} 







num_reMdx_JO_active_minusl 


i 
t 


ue(v) 


nu m_ref_idx_l 1 _act i ve_ mi mis 1 


i 
• 




wcighted_pred_flag 


i 
i 


U V 1 J 


weighted_bipred_explicit_flag 


1 




weighted_bipred_implicit_flag 


i 
i 


u( 1 ^ 


slice_qp_minus26 /* relative lo 2o */ 


1 
i 


v ) 


slice_qp_s_minus2o /* relative to zo */ 


| 


*ic( v^ 


redundant_slice_flag 


1 


urn 


«/■■■ ntr naromAf^rc f| *i o 

VUI pi v. |J«tl it 11 Id O 


1 


u(!) 


if( vui_pic_paramctcrs_flag ) ( 






vui_pic_paramcicrs( ) 


i 




) 






rbsp_lrailing_biLs( ) 


1 




} 
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7.3.23 Supplemental enhancement information RBSP syntax 



sei_rbsp( ) { 


Category 


Descriptor 


do 






| sei_message( ) 


7 




while( more_rbsp_data( ) ) 






rbsp_trailing_bits( ) 


7 




) 







7.3.2.3.1 Supplemental enhancement information message syntax 



sei_message( ) { 


Category 


Descriptor 


PayloadType = 0 






wnile( next_bits( 8 ) = = OxFF ) { 






byte_ff /* equal to OxFF */ 


7 


"(8) 


PayloadType += 255 






) 






last_payload_type_byte 


7 


u(8) 


PayloadType += last_payload_type_byte 






PayloadSize = 0 






while( next_bits( 8 ) = = OxFF ) { 






byte_ff 


7 


u(8) 


PayloadSize += 255 






) 






last_payload_size_byte 


7 


u(8) 


PayloadSize += Iast_payload_size_byte 






sei_payload( PayloadType, PayloadSize ) 


7 




} 







7.3.2.4 Picture delimiter RBSP syntax 



pic_delimiter_rbsp( ) [ 


Category 


Descriptor 


three_reserved_bits 


8 


u(3) 


pic_type 


8 


"(3) 


non_stored_pic flag 


8 


u(l) 


rbsp trail ing_bits( ) 


8 




i 







7.3.2.5 Filler data RBSP syntax 



filler_data_rbsp( NumBytesInRBSP ) { 


Category 


Descriptor 


while( next_bits( 8 ) = = OxFF ) 






byte_ff 


9 


f(8) 


rbsp_trailing_bits( ) 


9 




i 
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7.3.2.6 Slice layer RBSP syntax 



slice_iayer_no_partilioning_rbsp( ) { 


Category 


Descriptor 


slice_header( ) 


4 




s!ice_data( ) /* all categories of slice_data( ) syntax */ 


4|5|6 




rbsp_slice_trailing_bits( ) 


4 




} 







Data partition RDSP syntax 
Data partition A RBSP syntax 



dpa_laycr_rbsp( ) { 


Category 


Descriptor 


slice_header( ) 


4 




slice_id 


4 


ue(v) 


slice_data( ) /* only the category 4 parts of slicc_daia( ) syntax */ 


4 




rbsp_slicc_irailing_bits( ) 


4 




) 







7.3.2.7.2 Data partition B RBSP syntax 



dpb_layer_rbsp( ) { 


Category 


Descriptor 


slicc_id 


5 


ue(v) 1 


slicc_data( ) /* only (he category 5 parts of slice_data( ) syntax */ 


5 




rbsp_slice_lrailing_bits( ) 


5 




) 







7.3.2.7.3 Data partition C RBSP syntax 



dpc_laycrjrbsp( ) { 


Category 


Descriptor 


slice_id 


6 


uc(v) 


slicc_data( ) /* only the category 6 parts of slice_data( ) syntax */ 


6 




rbsp_slice_trailing_bits( ) 


6 




) 







7.3.2.8 RBSP trailing bits syntax 



rsbp_lrai!ing_bits( ) { 


Category 


Descriptor 


rbsp_stop_bit /* equal to 1 */ 


All 


f(D 


while( !byle_aligncd( ) ) 






rbsp_alignment_bit /* equal to 0 */ 


All 


f(D 


) 
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7.3.2.9 RBSP slice trailing bits syntax 



rbsp_slice_trailing_bits( ) { 


Category 


Descriptor 


rbsp_stop_bit /* equal to 1 */ 


All 




if( entropy_coding_mode = = 1 ) 






wmie^ nexi^oits^ I ) = = } ) 






cabac_stuffing_bit /* equal to 1 */ 


All 


f(D 


while( !byte_ahgned( ) ) 






rbsp_alignment_bit /* equal to 0 */ 


All 


f(l) 


} 
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7.3.3 Slice header syntax 



slice_neader( ) { 


Category 


Descriptor 


pic_parameter_set_id 


4 


ue(v) 


frame_num 


4 


u(v) 


pic_structure 


4 


uc(v) 


fi rst_mb_i n_slice 


4 


u(v) 


slice_type_idc 


4 


uc(v) 


if( pic_order_cnt_iype = = 0 ) 






pic_order_cnt 


4 


u(v) 


else if( pic_order_cnt_ly|>c — = 1 ) 






delta_pic_order_cnt 


4 


sc(v) 


if( red undanl_slicc_ flag ) 






rcdundant_pic_cnt 


4 


uc(v) 


if( slice_lypejdc = = BiPred ) 






direct_spatiaLmv_pred_flag 


4 


u(l) 


num_reMdx_activc_override_flag 


4 


u(l) 


if( num_refJdx_activc_ovcrridc_flag ) { 






if( slicc_iype_idc == Pred 1 1 shce_typc_ide == SPred || 
slice typo idc = = BiPred ) ( 






nu m_ ref_idx_IO_a c ti v e_ mi n us 1 


4 




if( shce_type_tdc = = Birred ) 






num_rcf_idx_l I„active_minusl 


4 


ue(v) 


) 






) 






rcf_ idx_jreordering( ) 


4 




\\\ \ weignteu_prca_i lag otoc 

( ( slice_iypc_idc = = Prcd ) 1 1 ( slice_typc_idc = = SPrcd ) ) ) 1 1 
( wcighted_biprcd_explicit_flag && 
(slice type idc == BiPred))) 






pred_weight_table( ) 


4 




ref_pic_buffer_managemcnl( ) 


4 




slice_qp_dclta 


4 


se(v) 


let i«,r-.j% \Ai* ^F>rf»ri 1 1 cliff* Ivnf idc — — SIlllTa ^ i 

l\\ SllCC_lypC_lUC Ol rCU | J MILL V/P^* iXAl - k ?inua / | 






il( sliec_lypc_iac orrcu ; 






sp_for_switch_flag 


4 


u(l) 


si ice_q p_s_dcl ta 


4 


se(v) 


i 
1 






it( seflu_niter_pararncicrs_iuig i ) \ 






Gisauie_ QeoiocKiiiu unci » 


4 


u(!) 


if/ UiroKln /l/»HlrM" , k incr fllfpr flap 1 I 






slice_alpha_c0_offset_div2 


4 


se(v) 


slice_beta_offset_div2 


4 


se(v) 


} 






} 






if( nuni_sIice_groups_minusl >0 && 
mb_allocation_map_iyp^ >= 4 && 
mb allocation map type <= 6) 






slice_grou p_change_cycle 


4 


u(v) 


} 
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7.3.3.1 Reference index reordering syntax 



ref_idx_reordering( ) { 


Category 


Descriptor 


if( shce_typef ) != Intra && shce_typef ) 1= SIntra ) { 






ref_idx_reordering_flag_IO 


A 
H 


u(l) 


iff ref_Jdx_reordenng_flagJO ) { 






do { 






remapping_of_pic_nums_idc 


A 


ue(v) 


if( remapping_of_pic_nums_idc == 0 || 
rcmappin£_of pic nums idc = = 1 ) 






a bs_di ff_pic_num_ minus 1 


4 


ue(v) 


else if( remappin&_ofLpic_nums_idc = = 2 ) 






long_term_pic_idx 


4 


ue(v) 


} while( remapping_of_pic_nums_idc != 3 ) 






) 






) 






iff shce_type( ) = = BiPred ) { 






ref_idx_reordering_flagJl 


4 


u(l) 


iff ref_idx_reordering_flag_l 1 ) { 






do { 






remaDDine of Die mi mm irlr 


4 


ue(v) 


iff remapping of nic nums \clc Oil 

remapping_of pic nums idc = = I ) 






a bs_diff_pic_nu m_mi nus 1 


4 


ue(v) 


else iff remapping_oLpic_nums_idc = =2) 






long_term_pic_idx 


4 


ue(v) 


} whilef remapping_of_pic_nums_idc != 3 ) 






} 






) 






) 







7.3.3,2 Prediction weight table syntax 



pred_weight_table( ) { 


Category 


Descriptor 


luma_log_weight_denom 


4 


uefv) 


chroma_log_ weigh t_denom 


4 


uefv) 


for( i =0; i <= num_reOdxJ0_active_minus! ; i++ ) { 






lu ma_ weigh t_flag_10 


4 


ufl) 


if( Iuma_weight_flagJO ) { 






Iuma_weight_IO[ i ] 


4 


sefv) 


luma_offset_IO[ i ] 


4 


se(v) 


) 






chroma_weight_flagJO 


4 


ufl) 


iff chroma_weight_flagJO ) 






Mj-0;j<2;jH)( 






chroma_weight_10[ i ][ j ] 


4 


sefv) 


chroma_offset_10[ i ][ j ] 


4 


sefv) ] 


} 






} 
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\f( tvrW 1 = = BiPred } f 






fnrf i — 0- 1^= num ref idx 11 active minus Inum; i++ ) { 






111 [I Id lTClglll IMlg ■ *■ 


4 


u(l) 


iff Inmn weiffht flan 11 ^ ( 






lumA wpifht 11 T i 1 


4 


se(v) 


In ma offset 11 f i 1 


4 


se(v) 


I 






rhrnmA w^ifllt flaf? 11 
Lilt UI1M " rr 


4 


u(l) 


if( chroma woighi__flag__1 1 ) 






fort i = 0; i < 2: i++ ) 1 






chroma weight Il[ i J[ I ] 


4 


sc(v) 


chroma_ofFset_.ll[ i ][ j ] 


4 


sc(v) 


\ 
/ 






\ 
1 






num custom binred weichts 


4 


ue(v) 


for( i=0; i < num custom hi pred_ weights; i++ ) { 






iff num ret idx 10 active minusl>0) 






11 |J I" 


4 


xc(v) 


iff num rcf idx 11 active minus 1 >0) 






irp 11 


4 


xe(v) 


i«am«k ..minh f hinri»H I0r irn 10 If irn 111 
lunna weigiii uipitu "*l |v/ ji 11 \ 11 j 


4 


se(v) 


l„ m a wpifht binred Ilf irn 10 If irn 11 1 


4 


sc(v) 


Inma offset binred f irn 10 If irn 11 1 


4 


sc(v) 


rhrnma weight flae binredf irn 10 If irp 1 1 1 


4 


u(l) 


iff rhrnma weight fla< T binredf irn 10 If irp H ] ) 







fnrf i=0* i < 2* i++ ) ( 






chroma weight bipred_10[ irp_10 ][ irp_1 1 ][ j ] 


4 


se(v) 


chroma wei&ht binred llf irn K) If irn 11 If j 1 


4 


se(v) 


rhrnma offset biDredf irn 10 If irp 11 If i 1 


4 


se(v) 


1 
/ 






} 






} 






} 







ITU-T Rec. H.264 (2002 E) 

11 00641 A_l_> 



7.3.3.3 



Reference picture buffer management syntax 



ISO/IEC FCD 14496-10 



ref_pic_buffer_management( ) { 


Category 


Descriptor 


ref_pic_buffering_mode 


A 1 7 


u(l) 


if( ref_pic_buffering_mode = = 1 ) 






do { ; 






memory_management_control_operation 


4|7 


ue(v) 


if( memory_management_control_operation == 1 || 
memory management mnirnl nnAMfirm i \ 






difference_of_pic_nums_minusl 


4 1 7 




ue(v) 


if( memory_managernent_control_operation = = 2 1 1 






Iong_term_pic_idx 


4|7 


ue(v) 


if( memory_management_controLoperation = - 4 ) 






maxJong_term_pic_idx_phisl 


4|7 


ue(v) 


] while( memory_management_conlrol_operation 1= 0 && 
memory management control operation f - 5 ) 
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73,4 Slice data syntax 



cliff* f\'Al*Ji( \ i 

olid* U<lL*i^ ) \ 1 


Category 


Descriptor 


iff mh frame field adaDative Has && 

III 1111/ 1 1 fttuv nv>«v» |.#»»*" — 

( pic_structure = = 0 1 1 
nic structure = = 3 
pic structure = = 4 )) { 






MbPairY = first_mb_in_shce / (pic_width_in_mb_rmnusl + 1) 






MbPairX = first_mb_in_slicc % (pic_width_in_mb_rninusl + l) 






MbNum = ( MbPairY « 1 ) * (pic_width_in_mb_mmusl + !) 
+ MbPairX 






\ else 












do ( 






;iv oi;<-#» rvrW 1 Intra &&. slice tvnc( ) != SIntra ) 






If/ - anlrimu fY"wlin*J IlloHt* — ZZ () ) I 

lit eniropy__v.uuii»g__iiivut/ w / i 






■_rm.-a.tx ff\ mitt 

mo SKip_mn 


4 


ue(v) 


\/!rirf*n<iinFlao — more rbsn daia( ) 






j else i 






mb_skip_flag 


4 


ac(v) 


MoreDataFIag = !mb_skip_flag j 






) 






if/" Mf^ri-'n^fJlPHc* 

11^ rviui LUduu ' 






iff mh frame field adaptive fla& && 
( pic_structure = = 0 1 1 

pic_structure = = 3 1 1 pic_structure = = 4 ) && 
( slice type( ) ! = BiPred ) ) { 






IV/ MViMnm CSL O O ^ 

iiy IvlDlNlIm vC £. \J ) 






iviorieiuL'ecv.H.ii iij^r iu^v, — i 






il( MDrielciuecooingriag ) \ 






ww%ti f~* n\ rl r\ t*f\i\ 1 ncr ft 51 0 

mD_iicia_uecuuiiig ■ 


4 


u(l)|ae(v) 


KA hPw«lfir)i*rr»Hinc»Flao — () 
ivi nnioiuL'c.tAJviiiijii. *«o 






} else 






i-nh fu*IH dpf*rvlin< T fla** = 0 
iniJ_iiciu vicv^hjih^, ■ ■ ti o v ' 






1 






iff -iHintive hlock si/e transform Has* = = 0 ) 






mnrrnhlnck lavcrf ) 


4|5|6 




else 






mnrrnhlork laver abtf ) 


4|5|6 




if( entropy _c<>ding_mode = = 0 ) 






MoreDataFIag = more__rbsp_dala( ) 






else { 






if( MbNum < MAX_MB_ADDRESS ) { 






if( mb^frame_field_adapiivc_nag && 
( pic_structure = = 0 1 1 

pic structure = = 3 | 1 pic structure ; ococ 






T MbNum % 2 != 0 ) 






MoreDataFIag = 1 






else ( 






end__of_slice_flag 


4 


ae(v) 


MoreDataFIag = !cnd_of_slice_riag 
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} else 






MoreDataFIag = 0 






} 












J while( MoreDataFIag ) 






) 







NOTE - macrobIock_layer_abt( ) is specified in subclause 12.2.1 
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7.3.5 Macroblock layer syntax 



macroblock_layer( ) ( 


Category 


Descriptor 


mb type 


4 ; 


ue(v) | ac(v) 


if( num_mb_partition[ mb^type ] = = 4 ) 






sub_mb__prcd( mb_type ) 


4 




else 






mb_pred( mbjype ) 


A 
** 




SendRcsidual = 0 






if( mb_parlition_pred__mode( , I) == Intra && 

mo type - = intra ) / inu«j i oa i u_a_ i inu_ij|'v > 






SendRcsidual = 1 






else { 






cod ed_ bloc k_pa t tc rn 


4 


mc(v) | ae(v) 


if( coded_block_patlern > 0 ) 






ScndResidual = i 






I 






1 1 v jenoKusiuudi ; \ 






i#Y !rr»K frijivw* fi*»IH ;i<1:mliv<* flaw 1 

(mb_framc_field_adapiive_flag && 

( pic_structure - - 0 1 1 
, pic_structure = = 3 1 1 pic_structure = = 4 ) && 
first non skip mb in pair( ) ) 






delta_qp 


4 


sc(v) | ae(v) 


residual ( ) 


5|6 




} 






) 







7.3.5.1 Macroblock prediction syntax 



mb__prcd< mb_type ) { 


Category 


Descriptor 


if( mb_partilion_pred_modc( mb_lype, 1 ) = = Intra ) { 
if( mbjype == Intra_4x4 1 1 mb_type == SIntra_4x4 ) 




ce(v) | ae(v) 


for( i = 0; i < num_mbJntra_parlition( mb_type ); i++ ) /* for 
each 4x4 luma block */ 


intra_pred_mode 


4 
4 


intra_chroma_pred_mode 


ue(v) | ae(v) 


} else if( mbjype != Directs 16x1 6 ) { 






for( i = 0; i < num_mb_partilion( mb_Jype ); i++ ) 




if( num_refjdxj()_active_minusl > 0 && 

mb partition prcd mode( mb_type, i ) != Prcd_Ll ) 




ue(v) | ae(v) 


refjdxJO 


4 


for( i = 0; i < num_mb_partition( mb_type ); i++ ) { 




if( num_jef_idxjl_active_minusl >0 && 

mb partition prcd modc( mb type, i ) != Prcd_L0 ) 




ue(v) | ae(v) 


ref_idx_ll 


4 


fbr( i = 0; i < num_mb_partition( mb_Jype ); i++ ) { 




if( mb_partition_pred_mode ( mb_type, i ) != Pred_Ll ) 






for(j=0;j<2;j-H-) 






mvdJ0[i][j] 


4 


se(v) | ae(v) 


for( i = 0; i < num_mb_partition( mb_type ); i++ ) { 
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if( mb_partition_pred_mode( mb.type, i ) != Pred_L0 ) 






for(j =0;j <2;j++) 






mvd_Il[ i ][j ] 


4 


se(v) | ae(v) 


) 






} 







7.3.5.2 Sub macroblock prediction syntax 



sub_mb_pred( mb_type ) ( 


Category 


Descriptor 


for( i = 0; i < 4; i++ ) /* for each sub macroblock */ 






sub_mb_type[ i J 


4 


ue(v) | ae(v) 


IniraChromaPredModeFlag - 0 ~~ 






for( i = 0; i< 4; i++ ) /* for each sub macrobJock */ 






if( sub_mb_type[ i ] = = Intra 8x8 ) 






for( j = 0; j < num_sub_mb_intra_partition( sub_mb__typef i ] ); 
{ /* num sub mb intra partilion( ) = 4 */ 






intra pred mode 


4 


ce(v) | ae(v) 


IntraChromaPredModeFlae = J 






i 






if( IntraChromaPredModeFlae } 






intra_chroma pred mode 


4 


ue(v) [ ae(v) 


for( i _ 0; i< 4; i++ ) /* for each sub macroblock */ 






if( num_ref_idxJO_active_minusl > 0 && 
mb_type != Pred_8x8ref0 && 
sub_mb_type[ i ] != Intra_8x8 && 
sub_mb_type[ i ] != Direct_8x8 && 
sub_mb_pred_mode( sub mb type[ i ] ) != Pred Li ) 






refjdx_I0 


4 


ue(v) | ae(v) 


for( i = 0; i< 4; i++ ) /* for each sub macroblock */ 






if( num_ref_idxjl_active_minusl >0 && 
( sub_mb_type[ i ] != Intra_8x8 && 
sub_mb_type[ i ] != Direct_8x8 && 
sub_mb_pred mode( sub mb typef i ] ) != Pred L0 ) 






ref_idx 11 


4 


ue(v) | ae(v) 


for( i = 0; i < 4: i-M- ) /* for each «nh mnrmhlnrl */ 






if( sub_mb_type[ i ] != Intra_8x8 && 
sub_mb_type[ i ] != Direct_8x8 && 
sub_mb_pred mode( sub mb typef i ] ) != Pred LI ) 






for( j = 0; j < num sub mb partition( sub mb lvpcf i 1 )- j++ ) i 






for(k = 0;k<2;k++) lJ 






mvd I0[ i ][ j ][ k ] 


4 


se(v) | ae(v) 


for( i _ 0; i< 4; i++ ) /* for each sub macroblock */ 






if(sub_mb_type[ i ] != Intra_8x8 && 
sub_mb_ typef i ] != Direct_8x8 && 
sub_mb_pred mode( sub mb typef i ] ) 1= Pred LO ) 






for( j = 0; j < num sub mb partition( sub mb tvDef i 1 V i++ ^ 






for(k = 0;k<2;k++) ^ ' 






mvd jir i ir nr k i 


4 


se(v) | ae(v) 


) 
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73.53 Residual data syntax 



residual( mb_iype ) { 


Category 


Descriptor 


Iff iMitrnnv roflino mode — — I J 






rnviHual 4Y4.hlnclc — residual 4x4block cabac( ) 


5 1 6 




else 






rcsiduaL4x4block = residual_4x4block_cavlc( ) 
if( mb^iypo = = Intra_16xl6 ) 


5 16 




rcsidual_4x4block( intra 1 6x 1 6DC, 1 6 ) 


5 




fur( 18x8 = 0; i8x8 < 4; i8x8++ ) /* each luma 8x8 block */ 

for( i4x4 = 6; i4x4 < num_sub_biocks( ); i4x4++ ) /* each 4x4 
sub-block of block */ ..... 




, 


if/ ~^,lx.w! KI/vfL nitfj»rn X/ ( \ iXxX ^ ^ 

ii( cocieu oiocK_p*iiiorii oc ^ 1 io A ° ) > 




i 1( mb_type = = Intra_ I ox l o ) 






residual_4x4block( intra 1 6x 1 6 AC, 1 6 ) 


5 




else 






residual_4x4block( luma, 16 ) 


5 | 6 


if( codcd_block_pauern & UxjU ) / enroma uk~ resiuuai coueu / 


16 




for( iCbCr = 0; iCbCr < 2; iCbCr++ ) 
residual_4x4block( cnromaiJC h ) ) 




if( coded_bl(>ck_pattern & 0x20 ) /* chroma AC residual coded */ 
for( iCbCr = 0; iCbCr < 2; iCbCr++ ) 




for( i4x4 = 0; i4x4 < 4; i4x4++ ) 

residuaL4x4block( chromaAC, 16 ) 


5 |6 




} 
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residual_4x4block_cavlc( bIock_mode, MaxNumCoeff ) { 


Category 


Dp<£f* wi nfnr 
M-*x*%y\- m. ii/iui 


coeff_token 


5 16 
i 




NumCoeffs = total_coefr( coeff_token ) - trail ing_ones( coeff_token ) 






if( trai!ing_ones( coeffjoken ) > 0 ) 






for( 1 = trailing_ones( coeff_token i >= 0; i-- ) 






trailing_ones_sign[ i ] 


5 16 

J v 


ufl) 


if( tota!_coeff( coeff_token ) > 0 ) { 






for( i = NumCoeffs- 1 ; i >= 0; i- ) 






coefMevelf i ] 


516 


wc \ v J j 


if( totaLcoeff( coeffjoken ) < MaxNumCoeff) { 






total_zeros 


5 16 

i ^ 




i = total_coeff( coeff_token ) - 1 






ZerosLcft = ioial_zeros 






il( i>0 && ZerosLeft > 0 ) { 






do { 






run__before[ i ] 


5 1 6 


— — 

ce(v) 


ZerosLeft -= run_before[ i ] 






. 






} while( ZerosLeft > 0 && i > = 0 ) 






run__before[ i ] = ZerosLeft 






} 






j 






} 






} 







7.3,5.3.2 Residual 4x4 block CABAC syntax 



residuaL4x4block_cabac( BlockType, MaxNumCoeff) { 


Category 


Descriptor 


coded_block_flag 


5 |6 


ae(v) 


if( coded_biock_flag ) { 






for( i = 0; i< MaxNumCoeff - 1 ; i++ ) { 






significant_coeff_flag[ i ] 


5 | 6 


ae(v) 


it( significant_coeff_flag[ i ] ) { 






last_significant_coeff_flag[ i ] 


5|6 


ae(v) 


if( last_significant_coeff_flag[ i ] ) 






MaxNumCoeff = i + I 






} 






) 






coeff_absolute_value_minusl[ MaxNumCoeff- 1 ] 4 


5 1 6 


ae(v) 


coeff_sign[ MaxNumCoeff- 1 ] 


5|6 


ae(v) 


for( i = MaxNumCoeff-2; i >= 0; i- ) 






if( significant_coeff_flagf i ] ) { 






coeff_absolute_value_minusl[ i ] 


5 |6 


ae(v) 


coeff_sign[ i ] 


5 |6 


ae(v) 


) 






J 






} 
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7.4 Semantics 

7.4.1 NAL unit semantics 

NOTE - The Video Coding Layer (VCL) is specified to efficiently represent the content of the video data. The Network 
Abstraction Layer (NAL) is specified to format that data and provide header information in a manner appropriate for conveyance 
by the transport layers or storage media. All data are contained in NAL units, each of whicb contains an integer number o! bytes. 
A NAL unit specifies a generic format for use in both packet-oriented and bilslream systems. The format of NAL units for both 
packet-oriented transport and biLstream is identical except that each NAL unit can be preceded by a start code prefix in a 
bitstrcam-orienlcd transport layer. 

NurnByiesInNALunit specifics the si/.e of the NAL unit in bytes. This value is required for decoding of the NAL unit 
and shall be conveyed by externa! means. Framing of NAL units is necessary to enable inference of 
NumByteslnNALuntt. Such framing into a byte stream format is specified in Annex B and other methods for framing 
may be specified outside of this Recommendation | International Standard. 

NOTE - Any sequence of bits can be formatted into a sequence of bytes in a manner defined as an RBSP by suffixing the data with 
rbsp_trailin*_bits( )> and any RBSP can be encapsulated in a NAL unil a manner that prevents emulation of byte stream start code 
prefixes within ihc NAL unit. 

forbidden.bit shall be /.ero. 

NOTE - 'Hie forbidden Jril may be used by external specifications to signal potentially corrupt NAL units. 

nal.storagejdc equal to 0 signals that the content of the NAL unit belongs either to a picture that is not stored in the 
reference picture buffer, SEI data or Filler data, nal.storagcjdc shall not be 0 for sequence parameter set or picture 
parameter set NAL units. If naLstorageJde is 0 for one slice or data partition NAL unit of a particular picture, it shall be 
0 for all slice and data partition NAL units of the picture. naLstorageJde greater than 0 signals that the content of the 
NAL unil belongs to a decoded picture that is stored in the reference picture buffer. 

NOTE - In addition to signalling non-stored content, external specifications may use nal_storagc_idc to indicate the relative 
transport priority of the NAL unit in a manner not specified in this Recommendation | International Standard. The value 0 should 
be used to signal the lowest transport priority and the priority should grow in ascending order of na]_storage_idc values. 
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nal_unit_type indicates the type of element contained in the NAL unit according to the types specified in Table 7-1. 

Table 7-1 - NAL Unit Type Codes 



Value of nal_unit_type 


Content of NAL unit and RBSP syntax structure 


_ Category 


0x0 


Reserved for external use 




0x1 


Coded slice 

s J ice_l ay er_no_par ti ti on i ng_rbsp( ) 


4, 5,6 


0x2 


Coded data partition A (DPA) 
dpa_layer_rbsp( ) 


4 


0x3 


Coded data partition B (DPB) 
dpb_layer_rbsp( ) 


5 


0x4 


Coded data partition C (DPC) 
dpc_Iayer__rbsp( ) 


6 


0x5 


Coded slice of an IDR picture 
slice Jayer_no__partitionin£_rbsp( ) 


4,5 


0x6 


Supplemental Enhancement Information (SEI) 
sei_rbsp( ) 


7 


0x7 


Sequence Parameter Set (SPS) 
seq parameter set rbsp( ) 


0 


0x8 


Picture Parameter Set (PPS) 
pic_parameter_set_rbsp( ) 


1 


0x9 


Pistil rf» nv>l i mii / DT~\\ 

pic delimiter rbsp( ) 


8 


OxA 


Filler Data (FD) 
fi ller_data_rbsp( ) 


9 


OxB -0x17 


Reserved 




0x1 8 -Ox IF 


For external use 





An instantaneous decoder refresh picture (IDR picture) implies that all pictures in the multi-picture buffer are marked as 
unused except the current picture. Moreover, the maximum long-term index is reset to zero. An IDR picture contains 
only I or SI sl.ces, and IDR slice type shall be used for all slices of an IDR picture. 

rbsp[ i ] a raw byte sequence payload is defined as an ordered sequence of bytes that contains an SODB The RBSP 
contains the SODB in the following form: owe. i ne KBbF 

a) If the SODB is null, the RBSP is also null. 

b) Otherwise, the RBSP shall contain the SODB in the following form: 

1) The first byte of the RBSP shall contain the (most significant, left-most) eight bits of the SODB- the 
next byte of the RBSP shall contain the next eight bits of the SODB, etc.; until fewer than eight'bits 
of the SODB remain. 

2) The final byte of the RBSP shall have the following form: 

° T r h u fir o^? St si S nifkanl ' 'eft-most) bits of the final RBSP byte shall contain the remai ning bits 
ot the SODB, if any, 

ii) The next bit of the final RBSP byte shall consist of a single rbsp_stop_bit having the value one 
( 1 ), and 

iii) Any remaining bits of the final RBSP byte, if any, shall consist of one or more 
rbsp_alignment_bit having the value zero (*0*). 

The last byte of a RBSP shall never have the value zero (0x00), because it contains the rbsp_stop_biL 
If the boundaries of the RBSP are known, the decoder can extract the SODB from the RBSP by concatenating the bits of 
H e y one°ri f? and^ *?h- 0* < bs ^- bii > which '» ■« 0— significant, right-most) Saving the 
value zero (*0*) d,SCardmg a " y foI,ow 'ng fl«s significant, farther to the right) bits that follow it, which have the 
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Syntax structures having these RBSP properties are denoted in the syntax tables using an Lrbsp" suffix. These 
structures shall be carried within NAL units as the content of the rbsp[ i I data bytes. The association of the RBSP 
syntax structures to the NAL units shaii be as specified in Table 7-1. 

emulation_prevcntion_byte is a byte equal to 0x03. 

Within the NAL unit, an emulation_prevention_byte shall be present after an rbspf i ] byte having the value zero 
(0x00) if and only if a next byte of RBSP data rbsp[ i+1 ] follows that has one of the following four values: 

zero (0x00) 
one (0x01) 

- two (0x02) 

- three (0x03) 



NOTE - Example encoder procedure. The encapsulation of an SODB within an RBSP and the encapsulation of an RBSP wilhin a 
NAL unit is specified to prevent the emulation of start codes within NAL units while allowing any arbitrary SODB lo be 
represented within a NAL unit and enable identification of the end of the SODB within the NAL unit. 
The encoder can produce a NAL unit from an RBSP by the following procedure: 
The RBSP data is searched for byte-aligned bits or the following binary patterns: 

TXXXXXXX) OOOOOOxx' (where xx represents any 2 bit pattern: 00. 0L 10 or 1 1 ), 
and a byte having the value three (0x03) is inserted to replace these bit patterns with the patterns 

XXXXX3000 000000 i 1 (XXKXX)xx' 

The resulting sequence of bytes is then prefixed with the first byte of the NAL unit containing the indication of the type of RBSP 
data structure it contains. 

This process can allow any RBSP data to be sent in NAL unit while ensuring that no long SCP and no byte-aligned short SCP is 
emulated in the tyAL unit. 

7.4.2 Raw byte sequence payloads and RBSP trailing bits semantics 

7.4.2.1 Sequence parameter set RBSP semantics 

A sequence parameter set is called an active sequence parameter set when an IDR NAL unit refers to it. The parameters 
of an active sequence parameter set shall be replaced only when an IDR NAL unit refers to a different sequence 
parameter set. 

A picture parameter set includes the parameters that remain unchanged within a coded picture. Every picture parameter 
set shall refer to the active sequence parameter set. A decoded picture parameter set is an active picture parameter set 
when the first slice NAL unit or first DPA NAL unit of a coded picture refers to it. The picture parameters of an active 
picture parameter set shall be replaced only when the first slice NAL unit or DPA NAL unit of a picture refers to a 
different picture parameter set. 

NOTE - The sequence and picture parameter set mechanism decouples the transmission of infrequently changing information from 
the transmission of coded macroblock data. The sequence parameter set. the picture parameter set, and the slice header contain all 
the parameters needed to decode the slice data. It is recommended to convey sequence and picture parameter sets out-of-band 
using a reliable transport mechanism. However, if an application requires a self-contained bitstream, in -band parameter set 
information units may be used. In error-prone transmission environments, in-band sequence and picture parameter set information 
units should be protected in a way that assures their successful reception. Synchronization between in-band and out-of-band 
transmission of the sequence and picture parameter set information is outside of the scope of this Recommendation | international 
Standard. 

profile Jdc and leveMdc indicate profile and level as specified in Annex A. 
seq_parametcr_set_id identifies the sequence parameter set to be referred. 

Iog2_max_frame_num_minus4 specifics the MAX_FN used in frame number related arithmetic as follows: 

MAX_FN = 2 A ( log2_max_frame_num_minus4 + 4 ) (7-1) 

The value of Iog2_max_frarnc_num_minus4 shall be in the range of 0 to 12, inclusive. 

pic_order_cnt_type equal lo 0 or 1 indicates the method to code picture order count (see subclause 8.3.2). 
pic_ordcr_cnt_type values greater than 1 are reserved. 

Iog2 - max_pic_order_cnt_minus4 is used to specify the MAX_PIC_ORDER_CNT used in picture order count related 
arithmetic as follows: 

MAXJPIC_ORDER_CNT = 2 A ( log2_max_pic_order_cnt_minus4 + 4 ) - (7-2) 
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The size of the pic_order_cnt parameter in the slice header is log2_max_pic_order_cnt_minus4 + 4 bits The value of 
log2_max_pic_order_cnt_minus4 shall be in the range of 0 to 1 2, inclusive. 

offset_for_non_st red.pic indicates an expected picture order count difference of a non-stored picture compared to a 
stored picture having the same frame_num as the non-stored picture. ' 

nurn_stored_frames_in_pic_order_cnt_cycle signals the number of frame numbers in a picture order count cycle A 
picture order count cycle is a repetitive pattern of picture order count differences, each of which corresponds to a 
frame_num increment of one. H u 

offset_for_stored_frarne indicates an expected difference of picture order count corresponding to a frame num 
increment of one. Notation offset_for _stored_frame„ indicates the offse«_for_stored_rrame corresponding to index" n in 
the p.cture order count cycle. offset_for_stored_rrame_in_pic_order_cnt.cycIe is the sum of all values of 
onset_ior_stored_frame. 

num_of_ref_frames specifies the total number of short- and long-term pictures in the reference picture buffer. 
nL e ters red ~ frame " nMm " UPdate - behaViOUr CqUal l ° 1 SPCCifieS 3 SpedfiC deC ° der behavi ° ur case of missing frame 

£^7 dthJn -, ,nb r- lninU ^ ^ Pi c - he, '8 ht - in - m bs_minusl specify the size of the luma picture array internal to the 

S£fV" 7 ° maC :° b ° C , kS - 71,6 Pk,ure Wid ' h and hei8ht in units of macroblocks is computed by adding o he 
decoded values of each of these parameters. The maximum macroblock address, MAX MB ADDRESS shall be 
calculated according to Equation 7-3. ~ snaii oe 

MAX_MB_ADDRESS = (pic_width_in_mbs_minusl + l)x 

(pic_height_in_mbs_minusl + 1)- 1 (7.3^ 



The NUM_BITS_IN_MB ADDRESS indicates the number of bits used to code a macroblock address with a fixed- 
length unsigned integer. It is calculated as follows. 

If a picture is a field-structured picture or if macroblock-adaptive frame/field coding is not in use for the picture, then 

NUM_BITS_IN_MB_ ADDRESS = Cei l(Log2(MAX_MB_ADDRESS + 1 )) (7-4) 

otherwise, N UM_B ITS_IN_MB_ADDRESS shall be 

NUM_BITS_IN_MB_ADDRESS = Ceil(Log2(MAX_MB_ADDRESS + I ) - I ) ( 7 - 4a ) 

^S^il^S^^' 1 ^^ WhCther 3 SCl ° f Parame,CrS COmr0,li "S characteristics of the deblocking filter 

constrained intra_pred_nag equal to zero indicates normal intra prediction, whereas one indicates constrained intra 
prediction, where no intra prediction is done from macroblocks coded with mb_pred_type != Intra. 

l b ^M n, Vf e,d - ad ,f PtiVe - na ? ^ Ua ' ,0 Z6r ° indiC3teS n ° switchi "g between frame and field decoding mode at the 
macroblock layer, whereas one indicates the use of switching between frame and field decoding mode at the macroblock 

Ip P Tied q " ParameterS " nag eqUal 10 2er ° SpeC ' fies that default P»«"neier values for the vui sequence parameters shall be 
7.4.2.2 Picture parameter set RBSP semantics 

pic_parameter_set_id the picture parameter set identifier to be used for reference. 
seq_parameter_set_id refers to the sequence parameter set that is used with this picture parameter set. 

CARTr-r COd, " n ir d % e ^t^ A o r V ndiCaleS VLC a " d CAVLC (see subclause 91 >. wher «as value one indicates 
CABAC (see subclause 9.2). If CABAC is indicated, the ae(v) entropy coding is used for the assigned syntax elements. 

n?miT 3 - r T ,Uti0n eqUal 10 2er ° , indiCateS 1/4 ' Uma S3mp,e aCCUrate motion solution, and equal to one indicates 1/8 
luma sample accurate motion resolution. 
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adaptive block ske^ransformjlag equal to zero indicates usage of 4x4 transforms for the luma residual and equal 
one indicates usa"gc of transforms of size 4x4, 4x8, 8x4. and 8x8 for «he luma residual. Clause 12 specfies mod.f.cat.ons 
as indicated in clause 7 thai arc related to syntax, semantics, and decoding process. 

num_slice_groups_minusl the number of slice groups is equal to num_slicc_groups_minusl + 1. If 
num_slice_groups_minusl is zero, all slices of the picture belong to the same slice group. 

NOTE H)ne slice group means that no flexible macroblock ordering is applied. If num_siice_group S _minusl is greater than zero. 

flexible macroblock ordering is in use. 
mb all cation_map_type the macroblock allocation map type is present only if num_slice_groups_minusl is greater 
than 0. This parameter indicates how the macroblock allocation map i.s coded. The value of this syntax element shall be 
in the range of 0 to 6,. inclusive. 

mb_alIocation_map_type 0 is used lo indicate interleaved slices. 
mb_allocation_map_type I is used to indicate a dispersed macroblock allocation. 

mb_allocation_map_type 2 is used to explicitly assign a slice group lo each macroblock location in raster scan order. 
mb_allocation_map_type 3 is used to indicate one or more -foreground" slice groups and a "leftover" slice group. 
mb_allocation .map_typcs 4, 5 and 6 are used to indicate changing slice groups. num_slice_groups_minusl shall be I, 
when mb_allocation_map_lype is 4, 5 or 6. 

If mb allocation_map_type is 0, the runjength syntax clement follows for each slice group. It indicates the number of 
consecutive tnacroblocks that are assigned to the slice group in raster scan order. After the macroblocks ol the last slice 
group have been assigned, ihc process begins again from the first slice group. The process ends when all ihe macroblocks 
of a picture have been assigned. 

NOTE - Example To siunal macroblock row interleaving in a QCIF picture (where all even numbered macroblocks are in slice 
group 0. and all, odd numbered macroblocks are in slice group I), the number of slice groups i.s two and runjenglh .s I I tor both 
slice groups. 

If mb allocation_map_typc is I. the macroblock allocation map is formed using the following formula, where n is the 
number of columns in ihc picture (in macroblocks) and p is ihe number of slice groups to be coded. Specifically, 
macroblock position x is assigned to slice group S according to Equation 7-5. 

S(x) = ((x % n) + ((floor(x / n) * p) / 2)) % p (7-5) 

If mb_allocation_map_type is 2, slice_group_id identifies a slice group of a macroblock. 

NOTE - slice_group_id is repeated as often as there arc macroblocks in the picture. 
If mb_allocalion_map_iypc is 3. top_left_mb and bottom_right_mb arc specified for each slicc_group_id except for 
the last one The top_left_mb specifics the lop-lefl corner ol a rectangle and bottom_nghl_mb specifies the bottom-right 
corner lop_left_mb and bottom_right_mb are indicated as macroblock addresses. The foreground slice group contains 
the macroblocks that are within the indicated rectangle and that do not belong lo any slice group having a smaller 
sliccgroupjd The lasl slice_group_id is dedicated for the leftover slice group, which contains the macroblocks thai arc 
not covered by the foreground slice groups. The leftover slice group shall not be cmply. The size of the top_leh_mb and 
botlom_right_mb parameters NUM_BITS_IN_MB_ ADDRESS. 

If mb allocation_map_typc is 4, 5 or 6, mb_allocation_map_type and slicc_group_change_direction indicate ihe 
refined macroblock allocation map type according to Table 7-2. The macroblock allocation map is generated each time 
the decoder starts decoding or a new picture as described in subclause 8.3.4. 

Table 7-2- Refined macroblock allocation map type 



mb_allocation 


slice_group 


refined macroblock 


_map_type 


_change_direction 


allocation map type 


4 j 


o 


Box-out clockwise 


4 


1 


Box-oul counter-clockwise 


5 


0 


Raster scan 


5 


1 


Reverse raster scan 


6 


0 


Wipe right 


6 


! 


Wipe left 



slice _croup_change_rate_minusl is the minimum non-zero number of macroblocks by which the size a slice group can 
chanie from one picture to the next. The SLICE_GROUP_CHANGE_RATE variable is defined as follows: 
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SLICE_GROUP_CHANGE_RATE = slice^oup_change_rate_minus 1 + I (7 . 6) 

The decoded value of sJice_group_change_rate_minusl shall be in the range of 0 to MAX_MB_ADDRESS - 1 
inclusive. * 

num^ref JdxJO_active_minusl specifies the number of reference pictures minus 1 in the reference list 0 that are used 
to decode the picture. 

num_refjdxjl_active_minusl specifies the number of reference pictures minus 1 in the reference list 1 that are used 
to decode the picture. 

weighted_pred_flag equal to zero indicates that weighted prediction is not applied to P and SP slices 
weighted_pred_flag equal to one indicates that weighted prediction is applied to P and SP slices. 

weighted_bipred_explicit_nag equal to zero indicates that explicit weighted prediction is not applied to B slices 
weighted J>ipred_explicit_flag equal to one indicates that explicit weighted prediction is applied to B slices* 
weighted_bipred_explicit_flag shall be zero if weighted_bipred_implicit_flag is one. 

weighted_bipred_implicit_nag equal to zero indicates that implicit weighted prediction is not applied to B slices 
we,ghted_bipred_implicit_flag equal to one indicates that implicit weighted prediction is applied to B 
slices.weighted_bipred_implicit_flag shall be zero if weighted J>ipred_explicit_flag is one. 

slice_qp_minus26 specifies the value of the default QP Y for the macroblocks in an I, SI, P, SP, or B slice as specified in 
Equation 7-7. The value of this syntax element shall be in the range of -26 to +25, inclusive. 

slice r qp_s_ininus26 specifies the value of the default QS Y for the macroblocks in a SP or SI slice as specified in 
Eqatition 7-8. The value of this syntax element shall be in the range of -26 to +25, inclusive. 

redundant_slice_nag indicates the presence of the redundanl_pic_cnl parameter in all slice headers referencing the 
picture parameter set. 6 

vui pic_parameters_flag equal to zero specifies that default parameter values for the vui picture parameters shall be 
applied. 

7.4.2.3 Supplemental enhancement information RBSP semantics 

Supplemental Enhancement Information (SEI) contains information that is not necessary to decode VCL data correctly 
but is helpful for decoding or presentation purposes. 

7.4.2.3.1 Supplemental enhancement information message semantics 

An SEI NAL unit contains one or more SEI messages. Each SEI message consists of SEI header and SEI payload The 
type and size of the SEI payload are coded using an extensible syntax. The SEI payload size is indicated in bvtes SEI 
payload types are specified in Annex C. 

The SEI payload may have a SEI payload header. For example, a payload header may indicate to which picture the 
particular data belongs. The payload header shall be defined for each payload type separately. 

An SEI message is associated with the next slice or data partition RBSP in decoding order. 

byte_ff is a byte equal to OxFF identifying a need for a longer representation of the syntax structure it is used within. 
last_pay!oad_type_byte identifies the payload type of the last entry in an SEI message. 
iast_payIoad_size_byte identifies the size of the last entry in an SEI message. 
7.4.2.4 Picture delimiter RBSP semantics 

The picture delimiter may be used to signal the boundary between pictures, i.e., if present, it shall be inserted before the 
first NAL unit of a picture in decoding order. There is no normative decoder process associated with the picture 
delimiter. K 

pic_type signals which slice coding types are used in the following picture in decoding order. Table 7-3 shows the slice 
coding types that can occur in a picture with a given pic_type. 

Table 7-3- Meaning of pic_type 



pic_type 


Allowed slice_type_idc 


000 


Intra 
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001 


Intra, Pred 


010 


Intra, Pred, BiPred 


01 1 


S Intra 


100 


S Intra, SPred 


i nt 




I 10 


Intra, SIntra, Pred, SPred 


1 1 1 


Intra, Slnlra, Prcd ? SPred, BiPred 



If adaptive_block_si/.c_transfonn_flag = = I, pic_typc == '000', pic_typc == 4 00i\ and pic_type = = '010' arc 
allowed. 

non_stored_contcnt_flag equal to I indicates that the picture is not stored in the reference picture buffer. 

7.4.2.5 Filler data RBSP semantics 

The filler data RBSP contains bytes whose value shall be equal to OxFF. 

7.4.2.6 Slice layer RBSP semantics 

The slice layer RBSP consists of a slice header and slice data. 

7.4.2.7 Data partition RBSP semantics 

7.4.2.7.1 Data partition A RBSP semantics 

When data partitioning is in use, the coded data for a single slice is divided into three separate partitions. Partition A 
contains all symbols of Category 4. 

NOTE - Category 4 consists of the header symbols of all coded MBs. 

slicejd Each slice of a picture is associated a unique slice identifier within the picture. The first coded slice of the 
picture shall have identifier 0 and the identifier shall be incremented by one per each coded slice. 

7.4.2.7.2 Data partition B RBSP semantics 

When data partitioning is in use, the coded data for a single slice is divided into three separate partitions. Data partition B 
contains all symbols of Category 5. 

NOTE - Category 5 consists of the intra coded block patterns and coefficients. 

slicejd Each slice of a picture is associated a unique slice identifier within the picture. The first coded slice of the 
picture shall have identifier 0 and the identifier shall be incremented by one per each coded slice. 

7.4.2.7.3 Data partition C RBSP semantics 

When data partitioning is in use, the coded data for a single slice is divided into three separate partitions. Data partition C 
contains all symbols of Category 6. 

NOTE - Category 6 consists of the inter coded block patterns and coefficients. 

slicejd Each slice of a picture is associated a unique slice identifier within the picture. The first coded slice of the 
picture shall have identifier 0 and the identifier shall be incremented by one per each coded slice. 

7.3.2.8 RBSP trailing bits semantics 
rbsp_stop_bit is a single bit having the value one (T). 
rbsp_alignment_bit is a single bit having the value zero CO')- 

7.3.2.9 RBSP slice trailing bits semantics 
rbsp_stop_bit has the same semantics as in subclause 7.3.2.8. 
rbsp_alignment_bit has the same semantics as in subclause 7.3.2.8. 

cabac_stuffing_bit is a single bit having the value one (V)- When entropy_coding_modc is equal to I, the number of 
bins resulting from decoding the contents of the slice layer NAL unit shall not exceed 0.5*NumBytesInNALunit. A 
number of inserted cabac_stuffing_bil guarantee this condition. 

NOTE - The method for determining the number of inserted cabac_stuffing_btt at the encoder is as follows: First, at the beginning 
of the slice encoding, the index CodeLength pointing to the current position of the bit stream is stored in a variable 
CodeLengthStored, i.e., CodeLength <- CodeLengthStored. In addition, two counters EBins and EBinsx8 are set to zero. Then, 
each time a symbol is encoded, EBins is incremented by one, i.e., EBins <- EBins+1. In the renormalizatioruprocedure, each time 
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a new byte of compressed data is written, the following procedure is done: 
whi!e( EBins > 7 ) { 
EBins EBins-8 
EBinsx8 <- EBinsx8+l 

} 

After terminating the arithmetic encoding process such that the last pending bits have been written to the bitstream the number of 
written bits is determined by CodeLength <- 8*(CodeLength - CodeLengthStored). Given the number of encoded bins by 
EBins <- 8*EBinsx8+EBins and the number NumMbSIice of macroblocks per slice, the following procedure is done* 

EBins <- 2*EBins 

if (CodeLength >= 4*NumMbSlice) 

{ 

if (EBins > CodeLength) 

for(i = 0; i < EBins - CodeLength - I ; i-w-) 
putbitsC 1 *) /* writes cabac_stuffing_bit */ 

} 

7.4.3 Slice header semantics 

pic_parameter_set_id indicates the picture parameter set in use. 

frame_num labels the frame. frame_num shall be incremented by I for each coded picture in decoding order, in modulo 
MAX_FN operation, relative to the frame_num of the previous stored frame in decoding order. The framejnum serves 
as a unique ID for each frame stored in the reference picture buffer. Therefore, a frame cannot be kept in the buffer after 
its frame_num has been used by another frame unless it has been assigned a long-term frame index as specified below. 
No frame_num of a frame to be added to the reference picture buffer shall equal to any other among the short-term 
frames in the reference picture buffer. A decoder which encounters a frame number on a current frame having a value 
equal to the frame number of some other short-term stored frame in the reference picture buffer should Treat this 
condition as an error. 

pic_structure identifies the picture structure according to Table 7-4. 



Table 7-4 - Meaning of pic_structure 



Value of 
pic_structure 


Meaning 


0 


Progressive frame picture 


1 


Top field picture 


2 


Bottom field picture 


3 ! 


Interlaced frame picture, whose top 
field precedes its bottom field in time. 


! 4 


Interlaced frame picture, whose 
bottom field precedes its top field in 
time. 



Note that when top field and bottom field pictures are coded for a frame, the one that is decoded first is the one that 
occurs first in time. 

first_mbjn_slice specifies the macroblock address of the first macroblock contained in this slice. The size of the 
first_mbjn_slice parameter is NUM_BITS_IN_MB_ADDRESS. The value of first_mb_in_slice shall be in the ranee of 
0 to MAX JvlB__ADDRESS, inclusive. 

If macroblock-adaptive frame/field decoding is in use, first_mb_in_slice contains a macroblock pair address rather than a 
macroblock address and the number of macroblocks included in a slice shall be an even number. 

slice_type_idc indicates the coding type of the slice according to Table 7-5. 



Table 7-5 - Meaning of slice_type_idc 



Value of 


Prediction type of slice 


slice_type_idc 


(slice type) 


0 


Pred (P slice) 


1 


BiPred (B slice) 


2 


Intra (I slice) 


3 


SPred (SP slice) 


4 


SIntra (SI slice) 



5DOCID: <XP 1 100641 A_l_> 



ITU-T Rec. H.264 (2002 E) 



37 



ISO/IEC FCD 14496-10 

Tabic 7-6 specifies, which macroblock prediction types are allowed when a slice type is decoded. 

Table 7-6 - AH wed macroblock prediction types for slice__type_idc 



Prediction type of 
slice 
f slice tvnel 


Allowed 
macroblock 
prediction type 


Pred (P slice) 


Intra, Pred 


BiPrcd (B slice) 


Intra, Pred, BiPrcd 


Intra (1 slice) 


Inira 


SPrcd (SP slice) 


SPred, Intra 


SIntra (SI slice) 


SIntra, Intra 



If adapLive_bIock_size_transform_flag = = I, the use of SI slices and SP slices is not allowed. 

pic_ordcr_cnt carries the picture order count coded in modulo MAX_PIC_ORDER_CNT arithmetic. An IDR picture 
shall have pic_ordcr_cnt equal to 0. The size of the pic_order_cnl parameter is log2_max_pic_order_cnt_minus4 + 4 
bits. 

delta_pic_order_cnt signals the picture order count difference compared to the offset picture order count as described in 
subclause 8.3.2. 

redundant_pic_cnt is 0 for coded slices and data partitions belonging to the primary representation of the picture 
contents. The rcdundant_pic_cnt is greater than 0 for coded slices and data partitions that contain redundant coded 
representation of the picture contents. There should be no noticeable difference between the co-located areas of the 
decoded primary representation of the picture and any decoded redundant slices. Decoded slices having the same 
redundanl_pic_cnl shall not overlap. Decoded slices having a redundant_pic_cnt greater than 0 may not cover the entire 
picture area. 

direct_spatial_mv_pred_flag specifies the method used in the decoding process to determine the prediction values 
direct prediction. If dirccl_spalial_mv_pred_flag is set to 0, then direct mode motion parameters are calculated from the 
picture order count as described in subclause 10.3.3.2. Otherwise, if this flag is set to I, then direct mode motion 
parameters are calculated using the spatial motion vector prediction technique as described in subclause 10.3.3.1. 

mim_rcf_idx_active_override_flag equal to zero indicates that the nunwef_idx_H)_active_minusl and 
num_rcf_idx_l l_active_minus I specified in the referred picture parameter set are in effect. 
num_refJdx_aclive_ovcrride_flag equal to one indicates that the num_rcfJdx_IO_active_minus 1 and 
num_rcMdx_ll_aclive_rninusl specified in the referred picture parameter set are overridden by the following values in 
the slice header. 

num_rcf_idx_IO_active_minusl specifics the number of reference pictures minus I in the reference picture list 0 that 
are used to decode the picture. 

num_ref_idx_Il_activc_minnsl specifies the number of reference pictures minus 1 in the reference picture list I that 
are used to decode the picture. 

slice_qp_delta specifies the value of the QP Y for the MBs in the slice unless modified by the value of delta_qp in the 
macroblock layer. From this value, the initial QP Y parameter for the slice is computed as: 

QP Y = 26 + slice_qp_minus26 + slice_qp_delta (7-7) 

The initial decoded QP Y parameter shall be in the range of 0 to 51, inclusive. The value of QP Y is initialised to the above 
result and this value is used for the decoding of each macroblock in the slice unless updated by a delta_qp sent in the 
macroblock layer. 

sp_for_switchJflag indicates the decoding process to be used to decode the SP slice. 
slice_qp_s_delta is signalled for SP and SI slices. The QS Y parameter for the slice is computed as: 

QS Y = 26 + slice_qp_s_minus26 + slice_qp_s_delta (7-8) 

The value of QS Y shall be in the range of 0 to 51, inclusive. This value of QS Y is used for the decoding of all 
macroblocks in the slice. 
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disable_deblocking_rilter_flag equal to zero specifes that the deblocking filter shall be applied to the edges controlled 
by the macroblocks within the current slice. If disable_deblocking_filter_flag is I, Filter_Offset_A and Filter_Offset B 
shaU both be inferred to be equal to -51. If not present in the slice header the value of this field shall be inferred to "be 

slice_alpha_c0_offset_div2 specifies the offset used in accessing the ALPHA and CO deblocking filter tables for 
filtering operations controlled by the macroblocks within the slice. The decoded value of this parameter shall be in the 
range from -6 to +6, inclusive. From this value, the offset that shall be applied when addressing these tables is computed 

asr 



Filter_Offset_A = slice_alpha_cO_offset_div2 « 1 
If not present in the slice header the value of this field shall be inferred to be zero unless disable_deblocking_fiJter_flag 

slice_beta offset_div2 specifies the offset used in accessing the BETA deblocking filter table for filtering operations 
controlled by the macroblocks within the slice. The decoded value of this parameter shall be in the range from -6 to +6 
inclusive. From this value, the offset that is applied when addressing the BETA table of the deblocking filter is computed 



as 



Filter_Offset_B = slice_beta_offset_div2 « 1 



If not present in the slice header the value of this field shall be inferred to be zero unless disable_deblocking_f,lter_nag 

slice_group_.change_cycIe x SLICE_GROUP_CHANGE_RATE indicates the number of macroblocks in slice group 0 

SUCE ™ e oiJp 0f r H ANrF PA C m7- Ch 7- CyCle • fie,d iS Cei, ( L °s2(Ceil(MAX_MB_LOCATION . 
SLICE_GROUP_CHANGE_RATE))). The maximum value of slice group change cycle is 
Ceil(MAX_MB_LOCATION SLICE_GROUP_CHANGE_RATE). 

7.4.3.1 Reference index reordering semantics 

The syntax elements remapping_of_pic_nums_idc, abs_diff_pic_num_minusl, and long_tcrm_pic_idx specify the 
change from the default reference index lists to the reference index lists used for decoding the slice. 

r K r -!?rr re0rderin8 - fla8J ° , indiC3teS WhClher ,he S * nlax elements remapping_of_pic_nu m s_idc 
abs_ditf_pic_num_mmus 1, and long_term_pic_idx are present for specifying the reference index list 0. 

refJdx_reordering_f1agJl has the same semantics as ref_idx_reordering_flag_IO except that the reordering of the 
reference index list 1 is specified instead of the reference index list 0. 

remapping_of_pic_nums_idc together with abs_diff_pic_num_minusl and long_ierm_pic_idx indicates which of the 
reference pictures are re-mapped. The restrictions and the mapping to the code number are specified in Table 7-7 The 
number of signalling remapping_of_pic_nums_idc is limited to the num_ref_idx_IO_active_minusI + I 

Table 7-7 - remapping_of_pic_nums_idc operations for re-mapping of reference pictures 



Value of 
remapping_of_pic_nums_idc 


Re-mapping Specified 


0 


abs_diff_pic_num_minusl is present and corresponds to a negative 
difference to add to a picture number prediction value 


1 


abs_diff_pic_num_rninusl is present and corresponds to a positive 
difference to add to a picture number prediction value 


2 


long_term_picjdx is present and specifies the long-term index for a 
reference picture 


3 


End loop for re-mapping of reference picture set relative indexing default 
order 



abs_diff pic num_minusl plus 1 indicates the absolute difference between the picture number of the picture beino 
remapped and the picture number prediction value. 
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long_term_pic_idx indicates the long-term picture index of the picture being remapped. In the case of frame- structured 
pictures, it shall be less than max_long_tcrm_pic_Jdx_plus I ; while in the case of field-structured pictures, it shall be less 
than 2 x maxJong_term_pic_idx_plus I. 

7.43.2 Reference picture buffer management semantics 

The synlax elements rcf_pic_buffering_modc, memory_management_control_operation, 

diffcrence_of_pic_nums_minusl ? long__term_pic_idx, and max_long_term_pic_idx_plusl specify the buffering of a 
stored decoded picture into the reference picture buffer. Further, the reference picture buffer can be modified by marking 
pictures as unused, by assigning long-term pictures indices and by reselling of the reference picture buffer. The syntax 
elements rcf_pic_buffering_mode, mcmory_management_coniroLopcration, difference_of_pic_nums_minusi; 
long_term_pic_idx, and max Jong_lerm_pic_idx_plus 1 shall be identical for all coded slices of a coded picture. 

ref_pic_buffering_mode specifics ihe buffering mode of the currently decoded picture and specifies how the reference 
picture buffer is modified after the current picture is decoded. The values for ref_pic_buffering_mode are specified in 
Table 7-8. 

Table 7-8 - Interpretation of ref_pic_buffering_mode 



Value of 

ref _pi c__b u fferi ng_ mode 


Reference picture buffering mode specified 


0 


Sliding window buffering mode; A simple buffering mode providing 
a first-in first-out mechanism for pictures that are noi assigned a 
long-term index 


1 


Adaptive buffering mode: A more flexible buffering mode than 
sliding window buffering mode providing syntax elements to specify 
marking pictures as unused, to assign long-term pictures indices, and 
to reset the reference picture buffer. 



memory_managemcnt_control_operation specifies a control operation to be applied to manage the reference picture 
buffer. The memory_managcmeni_conlro!_operation parameter is followed by data necessary for the operation specified 
by the value of memory_managcmcnt_controLoperation. mcmory_managcmenl_control_opcration commands do not 
affect the buffer contents or the decoding process for the decoding of the current frame. They specify the necessary 
buffer status for the decoding of subsequent coded pictures. The values and control operations associated with 
memory_managcment_conlroLoperation are defined in Table 7-9. 

If memory_management_controLoperalion is Reset, all frames and fields in the reference picture buffer (but not the 
current picture unless specified separately) shall be marked "unused" (including both short-term and long-term pictures). 
Moreover, the maximum long-term picture index shall be reset to z.ero. 

The frame height and width shall not change within the bitstream except within a picture containing a Reset 
mcmory_management_control_opcralion command. 

A "stored picture" shall not contain any mcmory_managcmcnt_control_operation command which marks that (entire) 
picture as "unused". If the current picture is non-stored picture, the value of the 

memory_management_controLoperation shall not contain any of the following types of 
memory_managemcnl_control_operaiion commands: 

a) A Reset meniory_management_control_opcralion command, 

b) Any mcmory_management_controLopcration command which marks any other picture (other than the 
current picture) as "unused" that has not also been marked as "unused" in the RPS layer of a prior stored 
picture, or 

c) Any memory_management_control_opcration command which assigns a long-term index to a picture that 
has not also been assigned the same long-term index in the RPS layer of a prior stored picture. 



Table 7-9 - Memory management control operation (memory_inanagement_controi_operation) values 



Value of 

memo ry_management_control_ope ration 


Memory Management Control Operation 


Associated Data Fields Following 


0 


End memory_management_control_opcration 
Loop 


None (end of RPS layer) 
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1 


Mark a short-term picture as "Unused" 


difference_of_pic_nums_minus 1 


2 


Mark a long-term prame as "Unused" 


long_term_pic_idx 


3 


Assign a long-term index lo a picture 


difference_of_pic_nums and 
Iong_term_pic_idx 


4 


Specify the maximum long-Ierm picture index 


max_Jong_term_pic _idx_plus 1 


5 


Reset 


None 



difference^of_pic^nurns_.minusl is used to assign a long-term index to a picture or to mark a short-term picture as 
unused". 1 

long_term_picjdx is used to assign a long-term index to a picture or to mark a long-term picture as "unused". 

max long_term_pic_idxj>]usl indicates the maximum index allowed for long-term reference frames (until receipt of 
another value of maxJong^term_pic_idx_plusl). The decoder shall initially infer that maxJong_term_pic idx olusl is 
0 until some other value has been received. " ~ 

7.4.3.3 Prediction weight table semantics 

luma_!©g_weight_denom is the binary logarithm of the denominator for all luma weighting factors. 

chroma Jog_weight_denom is the binary logarithm of the denominator for all chroma weighting factors. 

luma.weighMlagJO indicates whether weighting factors are present for the luma component of the list 0 prediction. 

Iuma_weightJ0[ i ] is the weighting factor applied to the luma prediction value for reference index i in list 0 prediction. 

Iuma_offsetj0[ i ] is the additive offset applied to the luma prediction value for reference index i in list 0 prediction. 

If luma_weight_flagJO is equal to zero, luma_weight_IO[ i ] shall be interpreted as equal to 2 ,umaJo£ - wcight - dCDOn, and 
luma_offset_10[ i ] shall be interpreted as equal to zero. 

chroma_weight_flagJO indicates whether weighting factors are present for the Cb and Cr components of the list 0 
prediction. 

chroma_weightJO[i][OJ is the weighting factor applied to the Cb prediction values for reference index i in list 0 
prediction. 

chroma_ofTsetJO[i K0] is the additive offset applied to the Cb prediction values for reference index i in list 0 
prediction. 

chroma_weight_!0[ i ]( 1 ] is the weighting factor applied to the Cr prediction values for reference index i in list 0 
prediction. 

chroma_ofTsetJ0[ i ][ 1 ] is the additive offset applied to the Cr prediction values for reference index i in list 0 
prediction. 

If chroma_weight_flagJO is equal to zero, chroma_weight_10[ i ] shall be interpreted as equal to 2 chroraa - lo ^ wci * h, - dcnom and 
chroma_offsetJO[ i ] shall be interpreted as equal to zero. 

Iuma_weight_flagjl indicates whether weighting factors are present for the luma component of the list 1 prediction. 

Iuma_weightjl[i] is the weighting factor applied to the luma prediction values for reference index i in list 1 
prediction. 

Iuma_offset_ll[ i ) is the additive offset applied to the luma prediction value for reference index i in list 1 prediction. 

If luma_weight_flagjl is equal to zero, luma.weightj 1 [ i ) shall be interpreted as equal to 2 ,umaJo& - wcigh, - <tenom and 
luma_offsetJ 1 [ i ] shall be interpreted as equal to zero. 

chroma_weight_flag.ll indicates whether weighting factors are present for the chroma component of the list 1 
prediction. 

chroma_weight_ll[ i ][ 0 ] is the weighting factor applied to the Cb prediction values for reference index i in list 1 
prediction. 

chroma_offseUl[i][0] is the additive offset applied to the Cb prediction values for reference index i in list J 
prediction. 
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chroma_weightJl[ i ][ I ] is the weighting factor applied to ihe Cr prediction values for reference index i in list 1 
prediction. 

chroma_ofiset_Jl[ i ][ 1] is the additive offset applied to the Cr prediction values for reference index i in list I 
prediction. 

If chroma_weight_flag_ll is equal to zero, chroma_weightJ 1 [ i ][ j | shall he interpreted as equal to 2 chro,n:l - ,os - wcie,M - dcnoni 
and chroma_offsctJ 1 [ i ][ j ] shall be interpreted as equal to zero. 

num_custom_bipred_weights is the number of custom weight and offset combinations sent for bi-prcdictivc weighting. 

irp_l0 is the index of the reference picture in list 0 for whieh a custom weight and offset combination is indicated for 
custom bi-predictive weighting. 

irpjl is the index of the reference picture in list 1 for which a custom weight and offset combination is indicated for 
eusiom bi-predictive weighting. 

luma_weight_bipred J0[ irpJO ][ irp J 1 ] is the weighting factor applied to the luma prediction value for reference 
index irp JO in listO when used with reference index irpjl in list I for bi-prediction. 

Iuma_weight_bipred_ll[ irpJO ][ irpjl ) is the weighting factor applied to the luma prediction value for reference 
index irp_ll in list 1 when used with reference index irp_10 in list 0 for bi-prediction. 

Iuma_ ffset_biprcd[ irp JO ][ irpjl ] is the additive offset applied to the luma prediction values when reference index 
irp_10 in list 0 is used with reference index irpj I in list 1 lor bi-prediction. 

chroma_weight_flag_biprcd[ irpJO ]( irpjl ] indicates whether custom weight and offset combinations are sent for Cb 
and Cr prediction when reference index irp JO in list 0 is used with reference index irpjl in list I for bi-prediction. 

chroma_weightJ>ipredJO[ irp JO ][ irpjl ][ 0 ] is the weighting factor applied to the Cb prediction value for reference 
index irp JO in list 0 when used with reference index irpjl in list I for bi-prediction. 

chroma_weightJ>ipredJl[ irp JO ][ irpjl ][ 0 ] is the weighting factor applied to the Cb prediction value for reference 
index irpjl in list I when used with reference index irp JO in list 0 for bi-prediction. 

chroma__offset_bipred[ irpJO ][ irpj I ][ 0] is the additive offset applied to the Cb prediction values when reference 
index irpJO in list 0 is used with reference index irpj I in list 1 for bi-prediction. 

chroma_weight_bipredJO[ irp JO ][ irpjl ][ 1 ] is the weighting factor applied to the Cr prediction value for reference 
index irp JO in list 0 when used with reference index irpj I in list 1 for bi-prediction. 

chroma_weightJ)ipredJl[ irp JO ][ irpjl ][ I ] is the weighting factor applied to the Cr prediction value for reference 
index irpj I in list 1 when used with reference index irp JO in list 0 for bi-prediction. 

chroma_offsetJbipred[ irpJO ][ irpjl ][ 1 ] is the additive offset applied to the Cr prediction values when reference 
index irpJO in list 0 is used with reference index irpjl in list 1 for bi-prediction. 

If chroma_weight_flasJ>ipred| irp JO || irpjl ] is zero, chroma_weight_bipred_IO[ irp JO H irpj I 1( j ) and 
chroma_wcighl_bipred[lrpJO ]| irpjl H j | shall be interpreted as equal to 2 chrw,w - k ^ wc,i;h, - ilc,,om and 
chroma_offsctJ>iprcd[ irp_IO ][ irpj 1 |[ j I shall be interpreted as equal to zero. 

For any combination of irp JO and irpj I that is not sent, the following values shall be inferred: 

luma_weight_biprcd_10[ irp JO |[ irpjl 1 = luma_weight_IO[ irp JO |, 

luma_weightj>iprcdjl[ irp JO || irpjl I = luma_weightj I [ irpjl J, 

luma_offsct_bipred[ irpJO U irpj I 1 = 

(1urna_offsctJ0[ irp JO | + luma_offsclJ 1[ irpjl ] + 1) » I 

chroma_wcight_biprcd J0[ irp JO ][ irpj 1 1C j 1 = chroma_weight J0[ irpJO )I j 1, 

- chroma_weighLbiprcdJl[ irp JO ][ irpj I 1[ j | = chroma_weightJ t [ irpj I II j I. 

chroma_offset_biprcd[ irpJ0][ irpjl ][ j | = 

(chroma_offsetJO[ irpJO ]f j 1 + chroma_offselJ 1 1 irpjl Jl j J + 1) » 1 

7.4.4 Slice data semantics 

mb_skip_nin indicates a number of consecutive macroblocks decoded as MbSkip macroblock type in P slices or 
Directs 16x1 6 macroblock type and no additional transform coefficients in B slices when eniropy_coding_mode = = 0. 

For mb_framc_Field_adapuve_flag = = l t refer to the MB scanning order in Figure 6-4 (subclause 6.3). 
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mb_skip_flag indicates thai the macroblock is decoded as MbSkip macroblock type in P slices or Direct 16x16 
macroblock type and no additional transform coefficients in B slices when entropy_coding_mode = = 1 ~ 

For the MbSkip macroblock type no further information about the macroblock is decoded. The motion vector for a 
MbSkip macroblock type shall be obtained as described in subclause 8.3.1.3. If pic_structure indicates a frame then the 
decoded frame with ref_idx_10 == 0, which either was decoded from a frame picture or is the union of two* decoded 
Held pictures shall be used as reference in motion compensation. If pic_structure indicates a field, then the decoded field 
Of the same parity (top or bottom) with ref_idx_10 = = 0, which was either decoded from a field picture or is part of the 
most recently decoded frame picture shall be used as reference in motion compensation For 
mb frame_field_adapt.ve_flag = = 1, a pair of macroblocks will be in frame mode only. If one of a pair of macroblocks 
.s skipped, .t should be m the same frame/field coding mode as another macroblock of the same pair. In field codin<- the 
SmTfielTpa'rty " C ° Py,ng C °"'° Cated macrob,ocl < from the most recently decoded I or P field of the 

,o o-„? ,d r de , C ° d i ng -? ag ^1!° I*™ indiCatCS th3t thC macroblock Pa" is decoded in frame decoding mode and equal 
to one indicates that the macroblock pair is decoded in field decoding mode. 

end_of slice_flag equal to 0 indicates that another macroblock is following, whereas equal to 1 indicates the end of the 
slice and that no further macroblock follows. end_of_s.ice_flag is only present if entropy coding_mode =1 Tfo r each 
macroblock except for the last macroblock of the picture. 

If the current picture is a frame-structured picture and mb_adaptive_frame_field Hag is 1, the number of coded 
macroblocks in the slice shall be an even number. OI coaea 



7.4.5 



Macroblock layer semantics 



mb_type indicates the macroblock types. The semantics of mb_type depend on the slice type. In the following for I 
slices, SI slices, P shces, SP slices, and B slices, tables and semantics are specified for the various macroblockTy^s. 

It m m hwf!° Ck f yP T S , f0r 1 S,iCCS .? re J ?P ecifled in Table 7 -'0- I' adaptive_block_si 2 e_,ransform flag = = 1 the 
macroblock types for I slices are specified in Table 12-1. ~ 6 ' 



Table 7-10 - Macroblock types for I slices 



Value of 
mb.type 


Name of mb_type 
in I slices 


mb_partition.pred_ 
mode(, 1) 


num_sub_blocks( ) 


num.mb.intra 
_partition( ) 


0 


Intra_4x4 


i Intra 


4 


16 


1 


Intra. 16xl6_0_0_0 


Intra 


4 


na 


2 


Intra. 16x1 6.1.0.0 


Intra 


4 


na 


3 


Intra. 1 6x1 6.2.0.0 


Intra 


4 


na 


4 


Intra. 16x1 6.3.0.0 


Intra 


4 


na 


5 


Intra_16x16_(M_0 


Intra 


4 


na 


6 


Intra_16xl6_J_J_0 


Intra 


4 


na 


7 


Intra_16xl6_2_l_0 


Intra 


4 


na 


8 


lntra_16xl6_3_1_0 


Intra 


4 


na 


9 


Imra_16xI6_0_2_O 


Intra 


4 


na 


10 


Intra_J6x!6_l_2_0 


Intra 


4 


na 


1 1 


Intra. 16xl6_2_2_0 


Intra 


4 


na 


12 


Intra. !6xl6_3_2_0 


Intra 


4 


na 


13 


Intra. 16x1 6.0.0.1 


Intra 


4 


na 


!4 


lntra.l6x!6.1_0.1 


Intra 


4 


na 


15 


Intra. 16x1 6.2.0.1 


Intra 


4 


na 


16 


Intra. 1 6x1 6.3.0.1 


Intra 




na 
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17 


lntra_l6x!G_0_l_l 


Intra 


4 


na 


18 


Intra_l6xl6_l_l_l 


Intra 


4 


na 


19 


Imra_l6xl6_2_1_l 


Intra 


4 


na 


20 


Intra_16xl6_3_l_l 


Intra 


4 


na 


21 


In<ra_l6xl6_0_2J 


Intra 


4 ; 


na 


22 


Intra_l6xl6_1_2_l 


Intra 


4 


na 


23 


lmraJ6xl6_2_2_l 


Intra 


4 


na 


24 


Inlra_l6xl6_3_2_l 


Intra 


4 


na 



The following semantics are assigned to the macroblock types in Table 7-10: 

• Intra_4x4: the macroblock is coded as Intra prediction type. 

• Intra 16xl6_x_y_z: x: Imode, y: nc, z: AC these modes refer to 16x16 Intra prediction type. Imode numbers 
from 6 and upwards represent 16x16 intra coding. 



The macroblock types lor SI slices are specified in Table 7- 1 1 and Table 7-10. The mb_lype value 0 is specified in Table 
7-11 and the mb_lype values 1 to 25 are specified in Table 7-10 by adding I to the value ol mb_type in Table 7-10. 
NOTE - If adaptive_block_size_trdnsformJag = = I . the use of SI slices is not allowed. 

Table 7-11- Macroblock type with value 0 for SI slices 



Value of 
mistype 


Name of mb_type 
SI slices 


mb_partition_pred_ 
modc( , 1) 


num_sub_bIocks( ) 


0 


SInira_4x4 


S Intra 


4 



The following semantics arc assigned to the macroblock types in Table 7- II : 
• SIntra_4x4: the macroblock is coded as SIntra prediction type. 



The macroblock types for P slices are specified in Table 7-12 and Table 7-10. The mb.typc values 0 to 4 are specified in 
Table 7-12 and the mb.type values 5 to 28 are specified in Table 7-10 by adding 5 to the value of mb_type in Table 7- 
10. 

The macroblock types for SP slices are specified in Table 7-12, Table 7-11, and Table 7-10. The mb.type values 0 to 4 
are specified in Table 7-12, the mb_type value 5 is specified in Table 7-1 1 by adding 5 to the value of mb_type in Tabic 
7-1 I, arid the mb_type values 6 to 29 are specified in Table 7-10 by adding 6 to the value of mb_typc in Table 7-10. 
NOTE - If adaptive^hlocOizeJransform_nag = = I, the use of SP slices is not allowed. 



Table 7-12 - Macroblock type values 0 to 4 for P and SP slices 



Value of 
mb_typc 


Name of mb_type 


num mb partition 
() 


m b_pa r ti tio n_p red_ 
mode( , 1) 


mb_partition_pred 
_modc( , 2) 


num_sub_ 
b!ocks( ) 


0 


Prcd_UM6xl6 


1 


Pred_L0 




4 


1 


Pred_L0_LJ0_16x8 


2 


Pred_L0 


Prcd_L0 


4 


2 


Pred_L0_L0_8x!6 


2 


Pred_L0 


Prcd_L0 


4 


3 


Prcd_8x8 


4 


Na 


na 


4 


4 


Pred_8x8rcR) 


4 


Na 


na 


na 












na 



The following semantics are assigned to the macroblock types in Table 7- 



44 ITU-T Rec. H.264 (2002 E) 

<XP 1 1 00641 A_l_> 



ISO/IEC FCD 14496-10 

• Pred_L0_ 16x16, Pred_L0_L0_ 1 6x8, Pred_L0_L0_8x 1 6, and Pred_8x8: the macroblock js predicted from a oast 
p.cture with luma block sizes 16x16, 16x8, 8x16, and 8x8, respectively, and the associated chroma blocks For 
the macroblock types NxM=16xl6, 16x8, and 8x16, a motion vector is provided for each NxM luma block and 
the associated chroma blocks. If N=M=8, for each 8x8 sub macroblock an additional syntax element is decoded 
which indicates in which type the corresponding sub macroblock is decoded (see subclause 7.4 5 2) Depending 
on N,M and the sub macroblock types modes there may be 1 to 16 sets of motion data elements for a 
macroblock. 

• Pred_8x8refO: same as Pred_8x8 but ref_idx_10 is not sent and set to 0 for all sub macroblocks. 

The macroblock types for B slices are specified in Table 7-13 and Table 7-10. The mbjype values 0 to 22 are specified 
m Table 7-13 and the mb_type values 23 to 57 are specified in Table 7-10 by adding 23 to the value of mb.type in Table 

Table 7-13 - Macroblock type values 0 to 22 for B slices 



Value ol 
mb_type 


[ Macroblock type 
j mb_type name 


num. mb_pa rti tion( 


mb_partition_pred_ 
mode( , 1) 


mb_partition_pred_ 
mode( , 2) 


num_sub_ 
blocks( ) 


0 


Directs 1 6x1 6 


1 


Direct 




4 


1 


Pred_L0_ 16xi6 


1 


Pred_L0 




4 


2 


BiPred_LI_l6xl6 


1 


Pred_Ll 




4 


3 


, BiPred_Bi_ 16x16 


1 


BiPred 




4 


4 


Pred_L0_L0_16x8 


2 


Pred_L0 


Pred_L0 


4 


5 


Pred_L0JL0_8xl6 


2 


Pred_L0 


Pred_L0 


4 


6 


J BiPred_Ll_Ll_16x8 


2 


Pred_Ll 


Pred_L1 


4 


7 


BiPred_Ll_Ll_8xl6 


2 


Pred_Ll 


Pred_Ll 


4 


8 


BiPred_L0_Ll_16x8 


2 


Pred_L0 


Pred.Ll 


4 


9 


BiPredJL0_Lt_8xl6 


2 


Pred_L0 


Pred_Ll 


4 


10 


• BiPred_LI_L0_16x8 


2 


Pred_Ll 


Pred_L0 


4 


1 1 


BiPred_L 1 _L0_8x 1 6 


2 


Pred_LI 


Pred_L0 


4 


12 , 


BiPred_L0_BL16x8 


2 


Pred_L0 


BiPred 


4 


13 j 


BiPred_L0_BL8xl6 


2 


Pred_L0 


BiPred 


4 


14 


B i Pred_L 1 JB i_ 1 6x 8 


2 


Pred_Ll 


BiPred 


4 


15 I 


BiPred_LI_Bi_8xl6 


2 


Pred_Ll 


BiPred 


4 




BiPred_BLL0_16x8 


2 


BiPred 


Prcd_L0 


4 


17 


BiPred_Bi_L0_8xl6 


2 


BiPred 


Pred_L0 


4 


18 ! 


BiPred_BiJL l_16x8 


2 


BiPred 


Pred.Ll 


4 


19 


BiPred_BLLl_8xI6 


2 


BiPred 


Pred.L! 


4 


20 


BiPred_BLBM6x8 


2 


BiPred 


BiPred 


4 


21 


BiPred_Bi_BL8xl6 


2 


BiPred 


BiPred 


4 


22 


BiPred_8x8 


4 


na 




na 



The following semantics are assigned to the macroblock types in Table 7-13: 
• Direct. 1 6x 1 6 type: no motion vector data is transmitted. 
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• BiPred_x_yJNxM, x,y=L0,Ll ,Bi: each NxM block of a macroblock is predicted by using distinct motion 
vectors, reference pictures, and prediction directions. As indicated in Tabic 11-11, three different macroblock 
types that differ in their prediction methods exist for the 16x16 mode. For the 16x8 and 8x16 macroblock types, 
nine different combinations of the prediction methods are possible. If a macroblock is coded in 8x8 mode, an 
additional codeword for each 8x8 sub-partition indicates the decomposition of the 8x8 block as well as the , 
chosen prediction direction (sec Table 1 1-2). 

• BiPred_8x8: the macroblock is partitioned into sub macroblocks. The coding of each sub macroblock is 
specified using sub_mb_lype. 

coded_block_pattcrn contains information which of the 8x8 blocks - luma and chroma - contain transform coefficients; 
An 8x8 block contains four 4x4 blocks. An indication that an 8x8 block contains coefficients means that one or more of 
the four 4x4 blocks within the 8x8 block contains coefficients 4 . The four least significant bits of codcd_block_pattern 
contain information on which of four 8x8 luma blocks in the macroblock contains nonzero coefficients. These four bits 
arc denoted as coded_block_paltcrnY. The ordering of 8x8 blocks is indicated in Figure 6-5. A bit equal to zero in 
position n of codod_block_pattcrn (binary representation) indicates that the corresponding 8x8 block has no coefficients 
and a bit equal to 1 indicates that the 8x8 block has one or more non-zero coefficients. For chroma, 3 possibilities arc 
specified in Table 7- 14. 



Table 7-14 - Specification of nc values 



Value of nc 


Description 


0 


All chroma coefficients arc 0. 


I 


One ore more chroma DC coefficients are non-zero. 
All chroma AC coefficients are 0. 


2 


Zero or more chroma DC coefficients are non-zero. 
One ore more chroma AC coefficients are non-zero. 



The value of codcd_block_patlern for a macroblock is given by codcd_block_pattern = coded_block_paUcrnY + 16 x nc 

The coded_block__patlem is indicated with a different codeword for macroblocks in P, SP, and B slices compared to 
macroblocks in I and SI slices. 

If adaptive_block_sizc_transform_flag = = I , the semantics for coded_bhx*_paltern are specified in subclause 12.3. 

delta_qp the value of QP Y can be changed in the macroblock layer by the parameter delta_qp. The dclta_qp parameter 
is present only for non-skipped macroblocks, as defined by: 

• If codcd_b!ock_pattern indicates that there are nonzero transform coefficients in the macroblock or 

• If the macroblock is 16x16 based intra coded 

Furthermore, when macroblock- level adaptive frame/field coding is in use ? delta_qp is present only for the first non- 
skipped macroblock of each macroblock pair. 

The decoded value of dclta_qp shall be in the range from -26 lo +25, inclusive. This specifies the value or QP Y in the 
range [0..5 1 ], as given by 

QP Y = (QP Y + delta_qp + 52') % 52 (7-9) 
7.4.5.1 Macroblock prediction semantics 

All samples of the macroblock are predicted. The prediction mode is signalled using the mb_type syntax element. 
Depending on the mb_typc syntax element, intra prediction information is signalled using intra_pred_mode or inter 
prediction information is signalled using the syntax elements ref_idx_10, ref_idxjl, mvdJO, and mvdjl. 

If adaptive j5lock_sizc_transformJlag == 1, modifications to the macroblock prediction semantics arc specified in 
subclause 12.2.1.1. 

The return value of num_mbJntra_partition( ) is always equal to 16 if adaptivcJsIc>ck_size_transform_flag = = 0. 

NOTE - If adaptivc_block_size_transform_nag = = I the semantics regarding the function num_mbJntra_partition( ) arc 
specified in subclause 12.2.1.1. 

intra_pred_mode indicates the intra prediction mode. 
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intra_chroma_pred_mode indicates the type of spatial prediction used for chroma whenever any Dart of the t.. ma 
macroblock is intra coded. This is shown in Table 7- 15. luma 



Table 7-15 - Relationship between intra_chroma_pred_mode and spatial prediction modes 



Value of 
intra_chroma_pred_mode 


Prediction Mode 


0 


DC 


1 


Horizontal 


2 


Vertical 


3 


Plane 



ref_idx JO, when present, indicates the reference picture to be used for prediction. 

If pic_structure indicates that the current picture is a frame picture, then the reference picture is a previous frame in list 0 
that was e,ther md.cated as a single frame picture or a frame that was indicated as two field pictures and has been 
reconstructed as a frame. Thus for frames the following table gives the reference frame: 

Code_num Reference frame 



0 The first frame in list 0 

1 The second frame in list 0 

2 The third frame in list 0 



If pic.structure indicates that the current picture is a field picture, then the reference picture is a previous field in list 0 
that was either coded as part of a frame-structured picture or coded as a field-structured picture. Thus for fields the 
following table gives the reference field: 



Code_num Reference field 

0 The first field in list 0 

1 The second field in list 0 

2 The third field in list 0 



If num_refJdxJO_active yninus 1 is equal to, 0, refJdxJO is not present. If num_reMdxJO_active_rninusl is equal to 
1 .only a single encoded bit is used to represent refJdxJO. If num_reMdxJO_aclive_rninusl is greater than 1, the value 
of ref_idx JO is represented by a decoded index. 

refjdxjl has the same semantics as refJdxJO, except that it is applied to the reference index list I. 

mvd JO if so indicated by mbjype, vector data for I - 16 blocks are transmitted. For every block a prediction is formed 
for the horizontal and vertical components of the motion vector. mvdJO signals the difference between the vector 
component to be used and this prediction. Motion vectors are allowed to point to samples outside the reference picture 
If a sample outside the reference picture is referred to in the prediction process, the nearest sample belonging to the 
picture (an edge or corner sample) shall be used. All fractional sample positions shall be interpolated as described in 
subclause 8 3.2 If a sample referred in the interpolation process (necessarily integer accuracy) is outside of the reference 
pic ure ,t shall be replaced by the nearest sample belonging to the picture (an edge or corner sample). Reconstructed 
motion vectors shall be clipped to ±19 integer samples outside of the picture. 

mvdjl has the same semantics as mvdJO, except that it is applied to the reference index list 1. 

7.4.5.2 Sub macroblock prediction semantics 

sub_mb_type indicates the sub macroblock types. 
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If adaptive^block.size^ransform.flag = = 1, modifications lo the sub macroblock prediction semantics are specified in 
subclause 12.2.1.2. 

The return value of num_sub_mbJntra_parlition( ) is always equal to 4 if adaptivc_block_size_transform_flag = = 0. 
NOTE - If adaptive^block_si/e_transform^nag = = 1 the semantics regarding the function niim_stib_mbJmra_partition( ) arc 
specified in subclause 12.2.1.1. 
The sub macroblock types for P macroblocks are defined in Tabic 7-16. 

Table 7-16 - Sub macroblock types in P macroblocks 



Value of 
sub_mb_type 


Name of 
sub_mb_type 


num_sub_mb_ 
partition( ) 


sub_mb_pred_ 
mode( ) 


nu m_.su b_ mb_ i n t ra 
_partition( ) 


num sub blocks 
() 


0 


Pred_L()_8x8 


1 


Pred_L0 


na 


4 


1 


Pred_L0_8x4 


2 


Pred.LO 


na 


4 


2 


Pred_L0_4x8 


2 


Pred_L0 


na 


4 


3 


Prcd_L0_4x4 


4 


Pred_L0 


4 


4 


4 


Intra_8x8 


na 


Intra 







The following semantics arc assigned to the sub macroblock types in Table 7-16: 

• Pred_L0_ XxY, X,Y=4,8 the corresponding partition of the sub macroblock is predicted from a past picture 
with luma block si/.c 8x4. 4x8, and 4x4, respectively, and the associated chroma blocks. A motion vector is 
transmitted for each NxM=8x4, 4x8, and 4x4 block. Depending on N and M, up to 4 motion vectors may be 
decoded for a sub macroblock, and thus up to 16 motion vectors maybe dedoced for a macroblock. 

• Intra_8x8 the 8x8 sub-partition is coded in intra mode. Inlra_8x8 shall not be present in SPred slices. 

The sub macroblock types for B macroblocks arc defined in Table 7-17. 

Tabic 7-17 - Sub macroblock types in B macroblocks 



Value of 
sub_mb_type 


Name of 
sub.mh type 


num_sub_ 
mb_partition( ) 


sub_mb_pred„mode( ) 


n u m_.su b_ mb_in tr a 
_partition( ) 


num_sub_blocks( ) 


0 


Oirect_8x8 


1 


Direct 


na 


4 


1 


Pred_L0_8x8 


1 


PnxLLO 


na 


4 


2 


BiPredLM_8x8 


1 


PrcdJJ 


na 


4 


3 


BiPred_Bi_8x8 


1 


BiPred 


na 


4 


4 


Pred_L0_8x4 


2 


Pred_L0 


na 


4 


5 


Prcd_L0_4xK 


2 


Pred.LO 


na 


4 


6 


BiPred_Ll_8x4 


2 


Prcd_Ll 


na 


4 


7 


BiPnxLL 1.4x8 


2 


Prcd_Ll 


na 


4 


8 


BiPrcd_BL8x4 


2 


BiPred 


na 


4 


9 


BiPrcd_BL4x8 


2 


BiPred 


na 


4 


10 


Prcd_L0_4x4 


4 


Prcd_L0 


na 


4 


1 1 


BiPrcd_Ll_4x4 


4 


Pred_Ll 


na 


4 


12 


BiPred_Bi_4x4 


4 


BiPred 


na 


4 


13 


Intra_8x8 


1 


Intra 


4 


4 



The following semantics are assigned to the sub macroblock types in Table 7-17: 
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• Pred_L0_XxY, X,Y=4,8, have the same semantics as in Table 7-16. 

• BiPred_Z_X_Y, Z=Ll,Bi, X,Y=4,8 

• Intra_8x8 has the same semantics as in Table 7- 1 6. 
7.4.5.3 Residual data semantics 

If adaptive_block_size_transform_flag == 1, modifications to the residual data semantics are specified in subclause 
12.2.1.3. 

The return value of num_sub_blocks( ) is always equal to 4 if adaptive_block_size_transform_flag = = 0. 

NOTE - If adaptive_block_size_transfbrm_flag == 1 the semantics regarding the function num_sub_blocks( ) are specified in 
subclause 12.2.1.3. 

7.4.5.3.1 Residual 4x4 block CAVLC semantics 



7.4.5.3.2 Residual 4x4 block CABAC semantics 

coded„block_flag indicates whether the block contains non-zero transform coefficients. If coded_bIock_flag is equal to 
0, the block contains no non-zero transform coefficients. If coded_block_flag is equal to 1, the block contains°at least one 
non-zero transform coefficient. 

significant_coeff_flag[i] indicates whether the transform coefficient at scanning position i is non-zero. If 
significant_coeff_flag[i] is equal to 0, the transform coefficient at scanning position i is equal to zero; if 
significant_coeff_flag[i] is equal to 1, the transform coefficient at position i has a non-zero value. 

last_significant_coeff_flag[i] indicates for the scanning position i whether there are non-zero transform coefficients for 
subsequent scanning positions i+1 to max_numcoefM . If all following transform coefficients (in scanning order) of the 
block have value equal to zero last_significant_coeff_flag[i] is equal to 1. If lasl_significant_coeff_flag[i) is equal to 0, 
there are further non-zero transform coefficients along the scanning path. 

coefr_absolute_value_minus_l is the absolute value of the transform coefficient minus 1. 

coeff_sign is the sign of the transform coefficient. A coeff_sign equal to 0 indicates a positive transform coefficient; a 
coeff_sign equal to 1 indicates a negative transform coefficient. 



8 Decoding process 

8.1 Ordering of decoding process 

A macroblock or sub-partition is decoded in the following order. 

1. Parsing of syntax elements using VLC/CAVLC (see subclause 9.1) or CABAC (see subclause 9.2) 

2. Motion compensation (see subclause 8.4) or Intra prediction (see subclause 8.5) 

3. Transform coefficient decoding (see subclause 8.6) 

4. Deblocking Filter (see subclause 8.7) 



8.2 NAL unit decoding 

8.2.1 NAL unit delivery and decoding order 

This subclause presents the requirements for the NAL unit deliver and decoding order. 

Decoders conforming to this Recommendation | International Standard shall be capable of receiving NAL units in 
decoding order. Systems conveying NAL unit streams conforming to this Recommendation | International Standard shall 
either 

1) Present NAL unit streams to the decoder in decoding order, or 

2) Provide a means to indicate the NAL unit decoding order to the decoder in the case of enhanced-capability 
decoders which may be capable of receiving or processing some NAL units in an out-of-order fashion. 
No such enhanced capability is defined or required herein for decoders conforming to this 
Recommendation | International Standard. 
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The decoding order of a sequence parameter set shall precede the decoding order of other NAL units that refer to that 
sequence parameter set. 

The decoding order of a picture parameter set shall precede the decoding order of other NAL units that refer to that 
picture parameter set.. 

A coded picture is called a primary coded picture if the redundanL_slicc_flag is 0 or if its slice headers contain 
rcdundant_pic_cnt equal to 0. 

The decoding order of coded slices and data partitions of a primary coded picture shall he contiguous relative to the 
decoding order of coded slices and data partitions of other primary coded pictures. 

The decoding order of coded slices and data partitions of a primary or redundant coded picture shall precede the 
decoding order of coded slices and data partitions of any other coded picture that uses the primary coded picture as a 
reference for inter-picture prediction. 

The decoding order of any coded slices or data partitions of a primary coded picture shall precede the decoding order of 
any redundant slices or data partitions containing coded data for the same macroblock locations represented in the slice 
or data partition for the primary deoded picture. 

The decoding order of any redundant coded slice or data partition shall precede the decoding order of the coded slices 
and data partitions of any other coded picture that uses the primary coded picture corresponding to the coded slice or data 
partition of the redundant coded slice or data partition as a reference for inter-picture prediction. 

The decoding order of slices and data partitions of primary coded pictures shall be non-decreasing in frame number 
order. The decoding order of the slices and data partitions of non-stored primary coded pictures shall be subsequent to 
the decoding order of the the slices and data partitions of the stored picture with the same frame number. If multiple 
primary coded pictures share the same frame number, the decoding order of the slices and data partitions of the non- 
stored pictures shall be in ascending order of redundant_pic_cnt. 

Depending on the profile in use, arbitrary slice ordering may or may not be allowed. If arbitrary slice ordering is allowed, 
the slices and data partitions of a primary coded picture may follow any decoding order relative to each other. If arbitrary 
slice ordering is not allowed, the decoding order of slices and data partitions of a primary coded picture shall be 
increasing in the raster scan order of the first macroblock of each slice, and the decoding order of data partition A of a 
coded slice shall precede the decoding order of data partition B of the same coded slice, and the decoding of data 
partition B of a coded slice shall precede the decoding order of data partition C of the same coded slice, and data 
partitions A, B, and C of a coded slice shall be contiguous in decoding order relative to the decoding order of any data 
partitions or non-partitioned slice data NAL units of other coded slices. 

The decoding order of SEI NAL units shall precede the decoding order of the slices and the slices and data partitions of 
the corresponding slice or data partition or coded picture or sequence of pictures to which the SEI NAL unit corresponds, 
and shall be subsequent to the decoding order of any SEI NAL units, slices, and data partitions of pictures that preced the 
corresponding coded picture in decoding order. 

The decoding order of a picture delimiter, if present, shall precede the decoding order of the SEI NAL units, slices and 
data partitions of the corresponding coded picture, and shall be subsequent to the decoding order of any SEI NAL units, 
slices, and data partitions of pictures that precede the corresponding coded picture in decoding order. 

8.2.2 Parameter set decoding 

The decoder maintains 16 sequence parameter set locations. For each 16 possible values of seq_parametcr_sct_id, the 
most recent decoded sequence parameter set is copied into the location referenced by seq_parameter_set_id immediately 
before the decoding of the next IDR picture. 

The decoder maintains 64 picture parameter set locations. For each 64 possible values of pic_paramcter_set_id, the most 
recent picture parameter set is copied into the location referenced by pic_parameler_sol_id immediately before the 
decoding of a slice or DPA belonging to the next coded picture. 

83 Slice decoding 

8.3.1 Detection of coded picture boundaries 

Decoding of a new picture is started from the slice to be decoded, herein called the current slice, if the slice is not a 
redundant slice and if one of the following conditions is truc: 

1 . The frame number of the current slice is different from the frame number of the previously decoded slice. 

2. The frame number of the current slice is the same as frame number of the previously decoded slice, the 
na!_storage_idc of the previously decoded slice is zero, and the nal_storage_idc of the current slice is non-zero. 
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3. The frame number of the current slice is the same as frame number of the previously decoded slice, 
pic_order_cnt_type is 0, and pic_order_cnt is different from the pic_order_cnt in the previously decoded slice. 

4. The frame number of the current slice is the same as the previously decoded slice, pic_order_cnt_type is 1 and 
delta_pic_order_cnt is different from the delta_pic_order_cnt in the previously decoded slice. 

8.3.2 Picture order count 

Each coded picture is associated with a picture order count, called PicOrderCnt, which is a 32-bit signed integer. An IDR 
picture shall have PicOrderCnt equal to 0. The PicOrderCnt of each stored picture shall be stored as long as the picture 
stays in the reference picture buffer. 

The decoder should treat a wraparound, underflow or overflow of pic_order_cnt, PicOrderCn tOf fset, FrameNumOffset 
and AbsFrameNum defined in subclauses 8.3.2. 1 and 8.3.2.2 as an error. 

8.3.2,1 Picture order count type 0 

If pic_order_cnt_type is 0, the decoder shall maintain a picture order count offset, called PicOrderCntOffset, which is a 
32-bit signed integer. The PicOrderCntOffset shall be zero for an IDR picture. 

If pic_order_cnt_type is 0 and if the decoding of a new picture is started, the PicOrderCntOffset is updated and the 
pic_order_cnt of the picture is calculated. The pic_order_cnt of the previous picture in decoding order is herein called 
PreviousPicOrderCnt. If pic_order_cnt is smaller than PreviousPicOrderCnt and if (PreviousPicOrderCnt - 
pic_order_cnt) is greater than or equal to (MAX_PIC_ORDER_CNT / 2), PicOrderCntOffset is calculated according to 
Equation 8-1. 

PicOrderCntOffset = PicOrderCntOffset + MAX_PIC_ORDER_CNT (8- 1 ) 

If pic_order_cnt is greater than PreviousPicOrderCnt and if (pic_order_cnt - PreviousPicOrderCnt) is greater than or 
equal to (MAX_PIC_ORDER_CNT / 2), PicOrderCntOffset is calculated according to Equation 8-2. 

PicOrderCntOffset = PicOrderCntOffset - MAX_PIC_ORDER„CNT (8-2) 
Otherwise, the value of PicOrderCntOffset is not changed. 

If pic_order_cnt_type is 0 and if the decoding of a new picture is started, the PicOrderCnt of the picture is calculated 
according to Equation 8-3. 

PicOrderCnt = PicOrderCntOffset + pic_order_cnt (8-3) 
8.3.2.2 Picture order count type 1 

If pic_order_cnt_type is 1, the decoder shall maintain a FrameNumOffset that is a 32-bit unsigned integer. The 
FrameNumOffset of an IDR picture shall be zero. 

If pic_order_cnt_type is I and if the decoding of a new picture is started, the PicOrderCnt of the picture is calculated. In 
the following, AbsFrameNum and PicOrderCntCycleCount are 32-bit unsigned integers, and the frame_num of the 
previous picture in decoding order is called PreviousFrameNum. First, the FrameNumOffset is updated. If frame_num is 
greater than or equal to the PreviousFrameNum, FrameNumOffset is unchanged. Otherwise, when frame_num is smaller 
than PreviousFrameNum, FrameNumOffset is calculated according to Equation 8-4. 

FrameNumOffset - FrameNumOffset + MAX_FN (8-4) 

Second, the AbsFrameNum is calculated according to Equation 8-5. 

AbsFrameNum = FrameNumOffset + frame_num (8-5) 

Third, the PicOrderCntCycleCount is calculated according to Equation 8-6. 

PicOrderCntCycleCount = AbsFrameNum / num_stored_framesJn_pic_order_cnt_cycle 

(8-6) 



Fourth, the FrameNu ml n PicOrderCn tCycle is calculated according to Equation 8-7. 
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FrameNumlnPicOrderCntCycle = AbsFrameNum % 
num_stored_frames_in_pic_order_cnt_cycle 

(8-7) 

In the following, the EXPECTED_DELTA_PER_PIC_ORDER_CNT_CYCLE is the sum of offset_for_stored_frame 
values. Finally, the PicOrdcrCnt of the picture is computed using the algorithm expressed in Equation 8-8. 

PicOrderCnt = PicOrderCntCycleCount x 

EXPECTED_DELTA_PER_PIC_ORDER_CNT_CYCLE 



for( i = 0; i <= FrameNumlnPicOrderCntCycle; i++ ) 
PicOrderCnt = PicOrderCnt + offset_for_stored_framei 



PicOrderCnt = PicOrderCnt + delta_pic_order_cnt 



if( nal_storage_idc = - 0 ) 
PicOrderCnt = PicOrderCnt + offset_for_non_stored_pic (8-8) 

8.3.3 Decoder process for redundant slices 

If the redundant_pic_cnt in the slice header of a coded slice is greater than 0, the decoder may discard the ctxlcd slice. If 
some of the samples in the decoded primary picture are incorrect and if the coded redundant slice can he correctly 
decoded, the decoder should replace the incorrect samples with the corresponding samples of the redundant decoded 
slice. 

8.3.4 Specification of macroblock allocation map 

If decoding of a new picture is started, if num_slicc_groups_minusl is greater than 0 and if mh_allocation_map_type is 
4, 5 or 6, a macroblock allocation map is generated. Slice group 0 has a growth order specified in ubclauses 8.3.4.1- 
8.3.4.4. The number of macroblocks in slice group 0 is equal to slice_group_change_cycle x 
SLICE_GROUP_CHANGE_RATE. This number of macroblock locations in the specified growth order is allocated for 
slice group 0. The rest of the macroblocks of the picture are allocated for slice group 1 . 

8.3.4.1 Allocation order for box-out 

Let H denote the number of coded macroblock rows of the picture and W denote the number of coded macroblocks 
columns of the picture. Macroblock locations are indicated with coordinates (x, y), where the top-left macroblock 
location of the picture has coordinates (0, 0) and the bottom-right macroblock location has coordinates (W - I, H - 1). 
The allocation order is created using a AllocalionDirection variable thai indicates the next macroblock location relative 
to the current one. AllocationDirection can have four values: (-1, 0), (K0),(0,-l) and (0, I). Furthermore, the left-most 
and right most macroblock columns allocated in the allocation order and the top-most and bottom-most macroblock rows 
allocated in the allocation order are stored in the variables Left, Right, Top, and Bottom respectively. For the box-out 
clockwise macroblock allocation map type, the first macroblock location in the allocation order is (x, y) = (W/2, H/2) 
and the initial AllocalionDirection is (-1, 0). For the counter-clockwise macroblock allocation map type, the llrsl 
macroblock location in allocation order is (x, y) = ((W - l)/2, (H - l)/2) and the initial AllocalionDirection is (0, I). At 
the beginning, Left = Right = x, and Top = Bottom = y. A subsequent macroblock location (x, y) in allocation order is 
allocated by searching the first row from top to bottom in Table 8- 1 for the same value of AllocationDirection and where 
the given condition is true. Then, the x, y, AllocationDirection, Left, Right, Top and Bottom variables are updated 
according to the refined macroblock allocation map type. If Left >= 0, Right < W, Top >= 0 and Bottom < H, the next 
macroblock location (x, y) in allocation order is allocated as described above. Otherwise, all macroblock locations have 
been allocated. 



Table 8-1 - Allocation order for the box-out macroblock map allocation type 



AllocationDirection 


Condition 


Box-out clockwise 


Box-out counter-clockwise 


(-L0) 


x > Left 


x = x - 1 


x = x - 1 
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(-1.0) 


I x = = 0 


I y - Top - ! 

1 Top = Top - 1 

1 AMocationDirection = (J, 0) 


y = Bottom + 1 j 
Bottom = Bottom + 1 
AllocationDirection = (1 , 0) I 


(-1,0) 


J x = = Left 


1 x = x - J 

Left = Left - 1 
1 AllocationDirection = (0, - J ) 


x = x —1 S 
Left = Left - J 
AllocationDirection = (0, 1) 


(1,0) 


J x < Right 


1 x = x + J 


x = x + I \ 


(1.0) 


x = = W- 1 


1 y = Bottom + ) 

Bottom = Bottom + 1 
1 AJlocationDirection = (-1,0) 


y = Top - 1 j 

Top = Top - 1 

AllocationDirection = (-1, 0) t 


(1.0) 


x = = Right 


x = x + 1 

Right = Right + 1 
I AJlocationDirection = (0, 1) 


x = x + 1 j 
Right = Right + 1 

AJlocationDirection = (0, - 1 ) f 


(0,-1) 


y >Top 


y = v - 1 


y = y- 1 I 


(0,-1) 


y = = o ~i 


x = Right + 1 
Right = Right + I 
AllocationDirection — (C\ \\ 


x = Left - 1 1 
Left = Left - 1 

AllocationDirection = (0, 1) 1 


(0.-1) 


y = = Top j 


V = V — 1 

Top = Top - 1 

AllocationDirection = (1.0) 


y = y-i ! 

Top = Top -1 

AllocationDirection = (- 1 , 0) 1 


(0. 1) | 


y < Bottom 


V = V + 1 


y = y + 1 j 


(0. 1) 


y = = H - ] J 


x = Left - I 
Left - Left - 1 

AllocationDirection = (0, -1) 


x = Right + 1 I 
Right = Right + 1 j 
AllocationDirection = (0, - 1 ) j 


(0. 1) 


y = = Bottom 


y = y + 1 

Bottom = Bottom + 1 
AllocationDirection = (-1 ,0) 


y = y+ i 

Bottom = Bottom + 1 
AllocationDirection - (I ,0) ! 



8.3.4.2 Allocation order for raster scan 

IZ l Hf t " Ster f SCan Sl !?l gr °" P ma croblock allocation map type, the first macrobloek in the allocation order is the top-left 
one of the picture, and the allocation order follows the raster scan order. P 

Sal 6 riJh, rSe raSt f e , r h SCan S,iCC g ™P "^roblock ^'location map type, the first macrobloek in the allocation order is the 
bottom-right one of the picture, and the allocation order follows the reverse raster scan order. 

8.3.4 3 Allocation order for wipe 

lZ i-Y 1 ^ 8 n° UP macr0block Nation map type, the first macrobloek in the allocation order is the top-left 

dl I ,h C " re - UO r\ OTdCT rUnS fr ° m *° P l ° b °" 0m - The next macrobloek after the bottom macroblc of a 

column is the top macrobloek of the column to the right of the previous column. acrooiocK ot a 

Shr^n^h 6 " S, , iCe ^ maCroblock allocation map type, the first macrobloek in the allocation order is the bottom- 
right one of the picture. The allocation order runs from bottom to top. The next macrobloek after the top macroblock oTa 
column is the bottom macroblock of the column to the left of the previous column. P maCr ° 0lock of a 

8.3.4.4 Allocation order for macroblock level adaptive frame and field coding 

Allocation order follows Figure 6-4 in subclause 6.2, instead of raster scan. 
8.3.5 Data partitioning 

When data partitioning is not used, coded slices start with a slice header and are followed by all the entronv coded 
symbols of Categories 4, 5, and 6 (see Category column in clause 7) of the macroblock ^£££S£3L ofte 

o?mio^ a ! a h Par f i0n H ng ,i S USCd ' r Cr ° bIOCk dMa ° f 3 S,ice is P««i»ioned in three partitions. Partition A contains a 
S^rv s P ? r , r entr ° Py C ° ded Symb ° ]S ° f CatCg0ry 4 - Partition B contains 3 partition B header all symbol" of 
f a ,h 8 a ^-, , COntamS 3 Par,hi ° n C header and 3,1 s y mbo,s of ^tegory 6. When data partition, ingTused 

each partmon ,s conveyed ,n its own NAL unit, which may be empty if no symbols of the respective Category 
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NOTE - Symbols of Category 5 are relevant to the decoding of intra coded texture information. Symbols of Category 6 are 
relevant to the decoding of residual data in Inter slices. Category 4 encompasses all other symbol types related to the decoding of 
macroblocks and their information is often denoted as header information. The Partition A header contains all the symbols of the 
slice header/and additionally a slice number that is used to associate the partitions B and C with the partition A. The partition B 
and C headers contain only the slice number which allows their association with the partition A of the slice 

8.3-6 Decoder process for management and use of the reference picture buffer 

83.6. 1 Genera! 

Intro to multi picture buffer. 

Decoder stores reference pictures as indicated in the bitstream. These are used for prediction. The buffer is divided into 
two independent buffers: the short term buffer and the long term buffer. Pictures can only remain in the short term buflcr 
for a finite duration, given by MAX_FN. Pictures can remain in the long term buffer until the next IDR picture, mmco 
commands are used to control the contents of these buffers. 

The decoder employs indices when referencing a picture for motion compensation on the macroblock layer. Default 
indices arc defined. These indices of pictures in the reference picture buffer arc rc-mapped onto newly specified indices 
according to the remapping_of_pic_nums_idc ? abs_diff_pic_num_minusl, and long_term_pic_idx fields. 

83.6.2 Picture Numbering 

Picture numbers are used in the decoding process for management and use of the reference picture buffer for both 
changing the default indices and for controlling the contents of the reference picture buffer using memory management 
control operations. 

In frame structured pictures, the picture number, PN, of a frame thai has frame number FN, is given by PN = FN 

In field structured pictures, the picture number, PN, of a field that has frame number FN. is given by PN = 2 x FN if the 
field is a top field/and is given by PN = 2 x FN + I if the field is a bottom field. 

Lona term picture numbers are also used in the decoding process for management and use of the reference picture buffer. 
I ong term picture numbers are used for both changing the default indices of pictures in the long term bullcr, and are 
used for transferring pictures from short term buffer to the long term buffer and for removing pictures from the long term 
buffer. 

in frame structured pictures, the long term picture number, LTPN, of a frame that has long-term frame index LTF1, is 
given by LTPN = LTFI 

In field structured pictures, the long term picture number, LTPN, of a field that has long-term frame index LTFI, is given 
by LTPN = 2 x LTFI if the field is a top field, and is given by LTPN = 2 x LTFI + I if the field is a bottom field. 

In frame-structured pictures, the parameter MAX_PN is defined to equal M AX_FN, and in field-structured pictures „ the 
parameter MAX_PN is defined to equal 2 x MAX_FN. 

8.3.6.3 Default index orders 

8.3.6.3. 1 General 

A reference index is a relative index into a list of reference indices to indicate which reference picture out of the 
reference picture buffer is used for motion compensation. When decoding a P or SP slice, there is one such list of 
reference indices, called the first reference indexjist. When decoding a B slice, there may be two reference indices used 
per block each pointing into a separate lists of reference indices which are called the first reference index list and second 
reference index list. 

The first reference index list and the second reference index list have default mappings to the pictures numbers in the 
reference picture buffer as defined below. 

8.3.6.3.2 Default index order for P and SP slices in frame-structured pictures 

The default index order for list 0 prediction of P and SP slices in frame-structured pictures (i.e., frames which have not 
been given a long-term index) to precede the long-term frames in the reference indexing order. Within the set of short- 
term frames, the default order is for the frames to be ordered starting with the most recently-decoded reference frame and 
procccdino through to the reference frame in the short-term buffer that was decoded first (i.e., in decreasing order of 
framc_num in the absence of wrapping of the framc_num value). Within the set of long-term frames, the default order is 
for the frames to be ordered starting with the frame with the smallest long-term index and proceeding up to the frame 
with largest long-term index. 
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A field that is stored in the short term or long term buffer for which the opposite parity field is not stored in the same 
buffer, are not .ncluded in the default .ndex order, shall not be remapped, and shall not be used for prediction in framT 
structured pictures. F 1 ,ram e- 

A field that is stored in the short term or long term buffer for which the opposite parity field is not stored in the same 
buffer, are not .ncluded in the default .ndex order, shall not be remapped, and shall not be used for prediction in framT 
structured pictures. y u " lrdme 

For example, assuming no wrap of the frame.num field, if the buffer contains three short-term frames with fnm, n„ m 
equal to 300, 302, and 303 and two long-term frames with long-term frame indices 0 and 3, ^^ZS^S^T 
default relative index 0 refers to the short-term frame with frame_num 303, 
default relative index 1 refers to the short-term frame with frame_num 302, 
default relative index 2 refers to the short-term frame with frame_num 300, 
default relative index 3 refers to the long-term frame with long-term frame index 0, and 
default relative index 4 refers to the long-term frame with long-term frame index 3. 
8.3.6.3.3 Default index order for P and SP slices in field-structured pictures 

In the case that the current picture is field-structured, each field of the stored reference pictures is identified as a separate 
reference p.cture w„h a unique index. Thus field structured pictures effectively have a, Teas, twice the number iSSSS 
available for referencing. The calculated decoding order of reference fields alternates between reference pictures o the 
same and opposite parity, starting with fields that have the same parity as the current field-structured picture Raure 8 6 

tzz d e o c f a ri * ,e,d in a fie ' d - s r tured pic,ure pair - whi,e Fi * ure 87 sh °- ** — ofConS , 

one field of a reference frame was neither decoded nor stored, the decoding order calculation shall ignore the missini 
field and instead index the next available stored reference field of the respective parity in decoding orde^r. When thTre Lrf 
no more fields of the respective parity in the short term buffer, default indices shall be allocated to the not yet indexed 
fie ds of the other parity starting with the most recently decoded such field and progressing to the firs, decoded suS 
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Figure 8-1 - Default reference field number assignment when the current picture is the first field coded in a frame 
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Figure 8-2 - Default reference field number assignment when the current picture is the second field coded in 

frame 



8.3-6.3.4 Default index order for B slices in frame-structured pictures 

The organisation of short term pictures in the default order for B slices depends on output order, as given by 
PicOrderCnt. 

The default index order for list 0 prediction of B slices in frame-structured pictures is for ihe short-term frames (i.e., 
frames which have not been given a long-term index) to precede the long-term frames in the reference indexing order. 
Within the set of short-term frames, the default order is lor the frames to be ordered starting with the decoded reference 
frame with the largest value of PicOrderCnt less than the value of PicOrderCnt of the current frame and proceeding 
through to the reference frame in the short-term buffer that has the smallest value of PicOrderCnt; and then the frame 
with the largest value of PicOrderCnt greater than the value of PicOrderCnt of the current frame and proceeding through 
to the reference frame in the short-term buffer that has the smallest value of PicOrderCnt greater than the value of 
PicOrderCnt of the current frame. Within the set of long-term frames, the default order is for the frames to be ordered 
starting with the frame with the smallest long-term index and proceeding up to the frame with the largest long-term 
index. 

The default index order for list 1 prediction of B slices in frame-structured pictures is for the short-term frames (i.e., 
frames which have not been given a long-term index) to precede the long-term frames in the reference indexing order. 
Within the set of short-term frames, the default order is for the frames to be ordered starting with the decoded reference 
frame with the largest value of PicOrderCnt and proceeding through to the reference frame in the short-term buffer that 
has the smallest value of PicOrderCnt. Within the set of long-term frames, the default order is for the frames to be 
ordered starting with the frame with the smallest long-term index and proceeding up to the frame with the largest long- 
term index. 

The ordinary default order defined in the previous paragraph shall be used as the default index order for list I prediction 
unless there is more than one reference picture in the set and the ordinary default index order for list 1 prediction is the 
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same as the default index order for list 0 prediction. In this exceptional case, the default index order for list J prediction 
shall be the ordinary default index order with the order of the first two pictures switched. 

A field that is stored in the short term or long term buffer for which the opposite parity Field is not stored in the same 
buffer, are not included in the default index order, shall not be remapped, and shall not be used for prediction in frame 
structured pictures. 

8.3.6-3.5 Default index order for B slices in field-structured pictures 

The default index order for list 0 and list 1 prediction of B slices in field-structured pictures is as for frame-structured 
pictures except that it is split between even indices for same-parity fields and odd indices for opposite-parity fields. 

8.3.6.4 Changing the default index orders 

8.3.6.4.1 General 

The syntax elements remapping_of_pic_nums_idc, abs_diff_pic_num_minusl, and long_term_pic_idx fields allow 
indexing into the reference picture buffer to be temporarily altered from the default index order for the decoding of the 
current slice. A remapping_of_pic_nums_idc "end loop" indication indicates the end of a list of re-ordering commands. 
The indices are assigned starting at zero and increasing by one for each remapping_of_pic_nums_idc field Pictures that 
are not re-mapped to a specific order by remapping_of_pic_nums_idc, shall follow after any pictures having a re-maooed 
order in the indexing scheme, following the default order amongst these non-re-mapped pictures. 

8.3.6.4.2 Changing the default index orders for short term pictures 

abs_diff_pic_num_minusl plus one indicates the absolute difference between the picture number of the picture being re- 
mapped and the prediction value. For the first occurence of the abs_diff_pic_num_minusl field in ref idx reordering,) 
the prediction value ,s the picture number of the current picture. For subsequent occurences of the 
abs_diff_p,c_num_minusl field in ref_idx_reordering(), the prediction value is the picture number of the picture that was 
re-mapped most recently using abs_diff_pic_num_minusl. 

The decoder shall determine the picture number of the picture being re-mapped, PNQ, in a manner mathematically 
equivalent to the following, where the picture number prediction is PNP. 

if (remapping_of_pic_nums_idc = = 0) 
{ /* a negative difference */ 

if (PNP - abs_dif f_pic_num_minusl < 0) 

PNQ = PNP - abs_dif f_pic_num_minusl + MAX_PN; 

else 

PNQ = PNP - abs_diff_pic_num_minusl ; 

} 

else 

{ /* a positive difference */ 

if (PNP + abs_dif f_pic_num_minusl > MAX PN-1) 

PNQ = PNP + abs_dif :f _pic_num_minusl - MAX_PN; 
else 

PNQ = PNP + abs_diff_pic_num_minusl; 

} 

The encoder shall control remappin^of_pic_nums_idc and abs_difLpic_num_minusl such that the decoded value of 
abs_diff_pic_num_minus1 shall not be greater than or equal to MAX_PN. 

If example implementation, the encoder may use the following process to determine values of 

abs_diff_pic_nunuminusl and remapping_of_pic_nurnsJdc to specify a re-mapped picture number in question, PN: 
if (remapping_of_pic_nums_idc = = o) 
{ /* a negative difference */ 

if (PNP - abs_dif f_pic_num_minusl < 0) 

PNQ = PNP - abs_dif f_pic_num_minusl + MAX__PN; 
else 

PNQ = PNP - abS_diff jicjnUlt^minUSl.- 
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} 

else 

{ /* a positive difference */ 

if (PNP + abs_dif f_pic_num_minusl > MAX_PN- 1 ) 

PNQ = PNP + abs_dif f jpic_num_minusl - MAX_PN; 
else 

PNQ = PNP + abs dif f _pic_num_miriusl ; 

} 

where abs( ) indicates an absolute value operation. 

remapping_of_pic_nums_idc is then determined by the sign of MDELTA. 

The prediction value used by any subsequent abs_diff_pic_num_minus 1 rc-mappings is not affected by 
long_term_pic_idx. 

8.3.6.4.3 Changing the default index orders for long term pictures 

The long_tcrm_picJdx field indicates the long term picture number of the long term picture being remapped. 

8.3.6.5 Overview of decoder process for reference picture buffer management 

The reference picture buffer consists of two independent parts: a short term buffer and a long term buffer. The decoder 
shall assume the initial size of the long term buffer to be 0, that is, it shall assume that max_long_term_picJdx_plusl is 
set to zero. 

The long term buffer has capacity to store max_long_term_idx_plusl frames. The usage of the long term buffer is 
constrained so that it has capacity for no more than max_long_tcrm_idx_plus 1 top fields and no more than 
max_long_tcrm.Jdx_plusl bottom fields. 

The remainder of the reference picture buffer is allocated to the short term buffer, which has capacity to store 
(num_of_ref_framcs - maxJong_term_idx_plusl ) frames. There is no further constraint on its capacity to store top and 
bottom fields. For example, the whole of the short term buffer could be used to store top fields. 

nal_sloragc_idc indicates whether the current picture is stored in the reference picture buffer. When nal_slorage_idc is 
equal to 0, the current picture is not stored in the reference picture buffer, otherwise it is stored in the reference picture 
buffer. 

If the current picture is stored in the reference picture buffer, the process used for storing is indicated by 
ref_pic_buffering_mode, which indicates either "Sliding Window", a first-in, first-out mechanism, or "Adaptive Memory 
Control", a customised adaptive buffering operation specified with memory_management_conlrol_operation commands. 

In frame structured pictures, memory_managcmcnt_controLoperation commands apply to both fields of the frame. 

In field structured pictures, mcmory_management_controLoperation commands apply to individual fields. 

8.3.6.6 Sliding window reference picture buffer management 
The "Sliding Window" buffering mode operates as follows. 

If there is sufficient "unused" capacity in the sho?t term buffer to store the current picture, the current picture is stored in 
the short term buffer and no pictures are removed from the short term buffer. 

Otherwise if the current picture is a field-structured picture, the short-term field with the largest default index, that is, 
field that has been in the short term buffer for the longest time, is marked "unused", thus creating sufficient capacity to 
store the current picture. The current picture is then stored in the short term buffer. 

Otherwise if the current picture is a frame-structured picture, default indices arc calculated as done when decoding a 
field-structured picture, and the short-term field with the largest default index is marked "unused". If there is still 
insufficient "unused" capacity in the short term buffer to store the current picture, the short-term field which now has the 
largest default index is also marked "unused". The current picture is then stored in the short term buffer. 

8.3.6.7 Adaptive Memory Control reference picture buffer management 
8.3.6.7.1 General 

The "Adaptive Memory Control" buffering mode allows specified pictures to be removed from either or both of the short 
and long term buffers, allows specified pictures to be moved from the short term buffer to the long term buffer, allows 
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specified pictures to be removed from the Jong term buffer, allows the number of long term pictures to be modified, and 
allows the whole buffer to be reset, by use of memory_management_controLoperation commands. 

memory_management_control_operation commands are processed in the order they occur in the bitstream, and are 
processed after the whole picture has been decoded. When all commands have been processed, storage of the current 
picture is considered. When naJ_storage_idc is equal to 0, die current picture is not stored in the reference picture buffer, 
otherwise it is stored in the short term buffer. memory_management_control_operation commands in the bitstream shall 
be such that when naLstorage_idc indicates that the current picture is to be stored, that there shall be sufficient "unused" 
capacity in the short term buffer to store the current picture. 

8.3.6.7.2 Removal of short term pictures 

If memory_management_controLoperation equals 1 (Mark a Short-Term Picture as "Unused"), a specified short term 
picture in the short term buffer is marked as "unused", if that picture has not already been marked as "unused". 

If the current decoded picture number is PNC, difference_of_pic_nuim_minusl is used in an operation mathematically 
equivalent to the following equations, to calculate, PNQ, the picture number of the short term picture to be marked as 
"unused". 

if (PNC < dif f erence_of_pic_nums_minusl ) 

PNQ = PNC - dif f erence_of j>ic_nums_minusl - 1 + MAX_PN; 
else 

PNQ = PNC - difference^of j)ic_nums_minusl - 1; 

Similarly, the encoder may compute the difference_of_pic_nums_minus! value to encode using the following relation: 
if (PNC < PNQ) 

dif f erence_of_pic_nums_minusl = PNC - PNQ - l + max PN; 
else 

difference_of jpic_nums__minusl = PNC - PNQ - l; 

8.3.6.7.3 Removal of long term pictures 

If memory^management.controLoperation equals 2 (Mark a Long-Term Picture as "Unused"), a specified long term 
picture in the long term buffer is marked as "unused", if that picture has not already been marked as "unused". 

The field long_term_pic_idx indicates the the long term picture number, LTPN, of the picture to be marked as "unused". 

NOTE: this use of long_term_pic_idx is different to its use when transferring short term pictures to the long term buffer. 

8.3.6.7.4 Transfer of short term pictures to the long term buffer 

If memory_management_controLoperation equals 3 (Assign a Long-Term Index to a Picture), a specified short term 
picture in the short term buffer is transferred to the long term buffer with a specified long-term index if that picture has 
not already been transferred to the long term buffer. If the picture specified in a long-term assignment operation is 
already associated with the required long_term_pic_idx, no action shall be taken by the decoder. The specified short term 
picture is no longer in the short term buffer following the processing of this command, and shall not be referenced at a 
later point in the bitstream by reference to its picture number. 

If another picture is already present in the long term buffer with the same long-term index as the specified long-term 
index, the other picture is marked as "unused". 

The picture in the short term buffer to be transferred is identified by its picture number, which is derived from 
difference_of_pic_nums_minusl as in 9.2.6.6.1. 

A top field in the short term buffer can only be transferred to the top field of a long term frame, and a bottom field in the 
short term buffer can only be transferred to the bottom field of a long term frame. The long term frame number of the 
frame into which the short term picture is transferred is given by long_term_pic_idx. 

long^term_pic_idx shall not be greater than maxJong^term_idx_plusl-l . If long^term_pic_idx does not satisfy this 
constraint, this condition should be treated by the decoder as an error. 

For error resilience, the bitstream may contain the same long-term index assignment operation or 
maxJong_term_idx_plusl specification message repeatedly. 

A bitstream shall not assign a long-term index to a short-term picture that has been marked as "unused" by the decoding 
process prior to the first such assignment message in the bitstream. A bitstream shall not assign-a long-term index to a 
picture number that has not been sent. 
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Once a long-term picture index has been assigned to a picture, the only potential subsequent use of the long term 
picture's picture number within the bitstream shall be in a repetition of the long-term index assignment. 
long_term_pic_idx becomes the unique ID for the life of a long term picture. 

83.6.7.5 Modification of the size of the long term buffer 

If memory_managemcnt_controi_or)cration equals 4 (Specify the Maximum Long-Term Frame Index), 
maxJong__term_pic_idx_plusl indicates the maximum index allowed for long-term reference frames (until receipt of 
another value of max_long_tcrm_pic_idx__plusI ). 

If nmxJong_term_pic_idx_plusl is smaller than its previous value, all frames in the long term buffer having indices, 
greater than maxJong_lerm_pic_idx_plusl - I shall be marked "unused". 

If max Jong_tcrm_picJdx_plusi is greater than its previous value, the capacity of the short term buffer is reduced by the 
same amount as the capacity of the long term buffer is increased. The memory_management_controLoperation 
commands in the bitstream shall be such that at the lime of processing this Command, the reduced capacity of the short 
term buffer shall be sufficient for the contents of the short term buffer. 

NOTE: maxJong_term_pic_idx_plusl can therefore be used to remove long term pictures from the long term buffer hut 
can not be used to remove short term pictures from the short term buffer. 

The frequency of transmitting maxJong_term_idx_plus1 is out of the scope of this Recommendation. However, the 
encoder should send an maxJong_term_idx_plusl parameter upon receiving an error message, such as an Intra request 
message. 

8.3.6.7.6 Buffer reset 

If memory_management_control_operation equals 5 (Reset), or the current picture is an IDR picture, all pictures in the 
short and long term buffers are marked as "unused", and maxJong_term_pic_idx_plusl is set to zero. 

8.3.6.8 Error resilience with reference picture buffer management 

If required jVame_num_update_bchaviour equals 1 the following picture buffer management behaviour shall be used. 

If the decoder identifies that pictures that should have been stored have not been decoded, by a gap in frame numbers, the 
decoder shall act as if the missing pictures had been inserted into the reference picture buffer using the "Sliding 
Window" buffering mode. An index for a missing picture is called an "invalid" index. The decoder should infer an 
unintentional picture loss if any "invalid" index is referred to in motion compensation or if an "invalid" index is re- 
mapped. 

If requircd_framc_num_update_behaviour equals 0, the decoder should infer an unintentional picture loss if one or 
several frame numbers are missing or if a picture not stored in the reference picture buffer is indicated in an 
abs_diff_pic_num_minusl or long_term_pic_idx field. 

Note: In case of an unintentional picture loss, the decoder may invoke some concealment process. If 
required_frarne_num_update_bchaviour equals 1 , the decoder may replace the picture corresponding to an "invalid" index with an 
error-concealed one and remove the "invalid" indication. Otherwise, the decoder may insert an error-concealed picture into the 
reference picture buffer assuming the "Sliding Window" buffering mode. Concealment may be conducted by copying the closest 
temporally preceding picture that is available in the reference picture buffer into the position or the missing picture. The temporal 
order of the short-term pictures in the reference picture buffer can be inferred from their picture numbers. In addition or instead, 
the decoder may send a forced intra update signal to the encoder by external means (for example. Recommendation H.245) if such 
external means is available, or the decoder may use external means or back-channel messages (for example, Recommendation 
H.245) to indicate the loss of pictures to the encoder if such external means is available. 



8.3.6.9 Decoding process for macroblock level frame/field adaptive coding 

When mb_ frame_field_adaplive_flag == I, the decoded frame is scanned on a macroblock pair by macroblock pair 
basis, as shown in Figure 6-4 (subclause 6.2). A macroblock pair can be decoded in either frame or field decoding mode. 
For frame decoding mode, a macroblock pair is decoded as two frame macroblocks, and each can be further divided into 
one of block patterns shown in Figure 6-4. For field coding mode, a macroblock pair is first split into one top-field 
macroblock and one bottom-field macroblock, as shown in Figure 8-3. The lop-field macroblock and the bottom-field 
macroblock are further divided into block patterns shown in Figure 6-5. Each macroblock in cither frame or field 
decoding mode can have a different mb_typc described in subclause 7.4.6. 
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Figure 8-3 - Split of a pair of macroblocks into one top-Held macroblock and one bottom-field macroblock. 

When mb fieId_decoding_flag = = 0, the top macroblock of a macroblock pair is decoded first, followed by the bottom 
macroblock, as shown in F.gure 6-4 (subclause 6.2). When mb_field_decoding_lla g = = 1, the top-field macroblock is 
i eC fonots ,rSt ' f0 " 0Wed ^ b ° WOm ' f,e,d macroblock (see Figure 6-4). A few specific rules/conventions are specified 

For intra prediction if a block/macroblock is in field decoding mode, its neighbouring samples in calculating the 
prediction shall be the neighbouring samples of the same field. 

As in frame decoding mode, the prediction mode of a 4x4 field block is decoded based upon the prediction modes of the 
(above and left) neighbouring blocks. For interior blocks of a field macroblock pair, the neighbouring blocks used in 
coding of intra prediction mode are the blocks above and left of the current block. For boundary blocks of a field 
macroblock pa.r the above or left neighbouring block may be in different macroblock pair that can be of either frame or 
field coding mode. The neighbouring blocks for these boundary blocks shall be as follows: 

- If the above or the left macroblock pair is also in field decoding mode, the neighbours of the boundary 
blocks in the current macroblock pair are in the same field of the above or the left macroblock pair. 

- If the above or the left macroblock pair is in frame decoding mode, the neighbours of the boundary blocks 
in the top (bottom) field macroblock are defined to be the corresponding blocks in the top (bottom) 
macroblock in the frame macroblock pair. 

- For macroblock pairs on the upper boundary of a slice, if the left macroblock pair is in frame decoding 
mode, then the intra mode prediction value used to predict a field macroblock shall be set to DC 
prediction. 

8.4 Motion compensation 

The motion compensation process generates motion compensated predictions for picture blocks using previously 
decoded reference pictures. The reference picture selection is described in subclauses 733 7351-2 7461 and 836- 
7. If pic_structure indicates a field picture, only the reference field indicated by the ref.idxJO or ref_idx_»l is'used in the 
motion compensation. The motion vectors to be used are described in subclauses 7.3.5. 1 -2, 7.4.5, 7.4.6. 1 and 8.4. 1 

If the current MB pair is in frame mode and one or more neighbouring blocks is in field mode, the reference frame index 
used in MV prediction from the field coded neighbours is obtained by dividing the reference field list index by 2 and 
truncation any fractional result toward zero to obtain the effective frame index for prediction. If the MB pair is in frame 
mode, the reference field number for any frame coded neighbour is obtained by multiplying the reference frame index by 

pTcture a buffer 0bl0Ck " ^ ^ ^ maCroblock ma y refer to < to P or bo «° m ) ^Id " the reference 
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8.4,1 Predict! n of vector components 

No vector component prediction takes place across macroblock boundaries of macroblocks that do not belong to the 
same slice. For the purpose of vector component prediction, macroblocks that do not belong to the same slice are treated 
as outside the picture. 

With exception of the 16x8 and 8x16 block shapes, "median prediction" (sec subclause 8.4.1.1) is used. In case the 
macroblock may be classified to have directional segmentation the prediction is defined in subclause 8.4.1.2. Motion 
vector for a Skip mode macroblock shall be obtained as described in subclause 8.4. 1 .3. 

8.4.1.1 Median prediction 

The prediction of the components of the motion vector value for a block E is formed based on the parameters of 
neighbouring blocks A, B, C and D as shown in Figure 8-4. This process is referred to as median prediction. 

A The block containing the sample to the left of the upper left sample in E 

B The block containing the sample just above the upper left sample in E 

C The block containing the sample above and to the right of the upper right sample in E 

D The block containing the sample above and to the left of the upper left sample in E 

NOTE - The prediction of A. B, C O and E may use different indices into the reference picture list. 




The following rules specify the predicted motion vector value resulting from the median prediction process for block E: 
If block C is outside the current picture or slice or is not available due to the decoding order within a 
macroblock as specified in Figure 6-4, its motion vector and reference picture index shall be considered 
equal to the motion vector and reference picture index for block D. 

If blocks B, C, and D are all outside the current picture or slice, their motion vector values and reference 
picture indices shall be considered as equal to the motion vector value and reference picture index for 
block A. 

If any predictor not specified by the first or second rules above is coded as intra or is outside the current 
picture or slice, its motion vector value shall be considered equal to zero and it shall be considered to have 
a different reference picture than block E. 

If only one of the three blocks A, B and C has the same reference picture as block E, then the predicted 
motion vector for block E shall be equal to the motion vector of the A, B, or C block with the same 
reference picture as block E; otherwise, each component of the predicted motion vector value for block E 
shall be the median of the corresponding motion vector component values for blocks A, B, and C. 

The following additional considerations apply in the case of macroblock-adaptive frame/field coding: 

If a block A is field type then it is assigned two frame MV's for the purpose of motion vector prediction. 
The first frame MV is the field MV of the block with vertical motion vector component multiplied by 2, 
and the second MV is the field MV of the block in same geometric location as A in the second MB of the 
MB pair (vertical motion vector component multiplied by two). If a block A is frame type then it is 
assigned two field MV's for the purpose of motion vector prediction. The first field MV is the frame MV 
of the block with vertical motion vector component divided by 2, and the second MV is the frame MV of 
the block in same geometric location as A in the second MB of the MB pair (vertical motion vector 
component divided by two). Similar rules arc used to determine the two reference frames (fields) of a field 
(frame) block. 
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- If E is in frame coding mode, the MVs of A, B, C and D used in calculating PMV are also frame-based If 
block A, B, C, or D is coded in field coding mode, its two frame-based MVs are averaged In that case the 
two reference field numbers of A, B, C or D shall be the same, and they shall be equal to the reference 
frame number of E multplied by 2. 

- If E is in field coding mode, the MVs of A, B, C and D used in calculating PMV are also field-based in 
the same field parity. If block A, B, C, or D is frame coded, the field-based motion vector is obtained by 
averaging the two field MVs of the block. In that case, two frame reference numbers of A B C or D shall 
be the same, and they shall be equal to the reference field number of E divided by 2 with 'truncation of 
fractional values toward zero. 

8.4.1.2 Directional segmentation prediction 

If the macroblock where the block to be predicted is coded in 16x8 or 8x16 mode, the prediction is generated as follows 
(refer to Figure 8-5 and the definitions of A, B, C, E above): 

a) Vector block size 8x 1 6: 

1) Left block: A is used as prediction if it has the same reference picture as E, otherwise "median 
prediction" is used 

2) Right block: C is used as prediction if it has the same reference picture as E, otherwise "median 
prediction" is used 

b) Vector block size 16x8: 

1) Upper block: B is used as prediction if it has the same reference picture as E, otherwise "median 
prediction" is used 

2) Lower block: A is used as prediction if it has the same reference picture as E, otherwise "median 
prediction" is used * 

If the indicated prediction block is outside the picture, the same substitution rules are applied as in the case of median 
prediction. 

For field-coded macroblocks, the directional segmentation follow the same conventions as the above but the 
neighbouring blocks are constructed from samples of the macroblock pair having the same field parity. 



8*16 16*8 




Figure 8-5 - Directional segmentation prediction 



8.4.1.3 Motion vector for a skip mode macroblock 

Motion vector for a Skip mode macroblock shall be obtained identically to the prediction motion vector for the 16x16 
macroblock type. However, if any of the conditions below hold, a zero motion vector shall be used instead: 

a) The Macroblock immediately above or to the left is not available (that is, is outside of the picture or 
belongs to a different slice) 

b) Either one of the motion vectors applying to samples A or B (as described in subclause 8.4.1.1) uses the 
last decoded picture as reference and has zero magnitude. 

8.4.1.4 Chroma vectors 

Chroma vectors are derived from the luma vectors. Since chroma has half resolution compared to luma, the chroma 
vectors are obtained by dividing the corresponding luma motion vectors by two. 

Due to the lower resolution of the chroma array relative to the luma array, a chroma vector applies to 1/4 as many 
samples as the luma vector. For example if the luma vector applies to 8x16 luma samples, the corresponding chroma 
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vector applies to 4x8 chroma samples and if the luma vector applies to 4x4 luma samples, the corresponding chroma 
vector applies to 2x2 chroma samples. 

8.4.2 Fractional sample accuracy 

Fractional sample accuracy is indicated by motion_resolution. If motion_resolution has the value 0, quarter- sample 
interpolation with a 6-tap niter is applied to the luma samples in the block. If motion_resolution has the value I, eighth- 
sample interpolation with an 8-tap filter is used. The prediction process for chroma samples in both cases is described in 
subclause 8.4.2.3. 



8.4.2.1 



Quarter sample luma interpolation 



In Figure 8-6, the positions labelled with upper-case letters wilhin shaded blocks represent reference picture samples at 
integer sample positions, and the positions labelled with lower-case letters within un-shaded blocks represent reference 
picture samples at fractional sample positions. 









f5*;| | aa | j|§>] 























c 



Ibbj 



E. 



[ddj 



G 


a 


b 


c 


m 


d 


e 


f 


9 




h 




] 


k 


m ! 


n 


P 


q 

















iee 



! 99 ! 



Ihhi 



Figure 8-6 - Integer samples (shaded blocks with upper-case letters) and fractional sample positions (un-shaded 
blocks with lower-case letters) for quarter sample luma interpolation. 



The luma prediction values al half sample positions shall be obtained by applying a 6-tap filter with tap values (1,-5, 20, 
20, -5, I). The luma prediction values at quarter sample positions shall be obtained by averaging samples at integer and 
half sample positions. The process for each fractional position is described below. 

The samples at half sample positions labelled *b* shall be obtained by first calculating intermediate values 
denoted as *b by applying the 6-tap filler to the nearest integer position samples in the horizontal 
direction. The samples al half sample positions labelled 4 h* shall be obtained by first calculating 
intermediate values denoted as % ti by applying the 6-tap filter to the nearest integer position samples in the 
vertical direction: 

fc=(E-5F+20G+20H-5I+J), 

/r=(A-5C+20G+20M-5R+T). 

The final prediction values shall be calculated using: 
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b=Clipl«>+16)»5), 
h=Clipl((A+16)»5). 

- The samples at half sample position labelled as *j' shall be obtained by first calculating intermediate A,alue 
denoted as y by applying the 6-tap filter to the intermediate values of the closest half sample positions in 
either the horizontal or vertical direction because these yield an equivalent result. 

j=cc-5dd+20h+20m-5ee+ff, or 
J=aa-5bb+20b+20s-5gg+hh, 

.where intermediate values denoted as <aa', 'bb\ >gg\ V and 'hh' shall be obtained by applying the 6-tap 
filter horizontally in an equivalent manner to 'b' and intermediate values denoted as 'cc' 'dd ' 'ee' W 
and \'ff shall be obtained by applying the 6-tap filter vertically in an equivalent manner to '/,' ' The' final 
prediction value shall be calculated using: j=Clipl((/+5 12)»10). 

- The samples at quarter sample positions labelled as 'a', V, 'd\ V, T, T, V and 'q' shall be obtained 
by averaging with truncation the two nearest samples at integer and half sample positions using- 
JSSST. C=(H+b)>> '' d =< G+h » 1 ' n = (M + h)»l, f=(b + j)»l, i = (h + j)», P kKj + m)». and 

- The samples at quarter sample positions labelled as V. «g' and 'p' shall be obtained by averao inR with 
g=(^m)>>i e p ^(L n s>>? SamP ' eS 31 h3,f SamplC P ° SitionS in ,he dia S° nal dircction ^ing e=^h»l, 

- The sample at quarter sample position labelled as V shall be obtained by averaging with rounding using 
tne tour nearest samples at integer positions using r=(G+H+M+N+2)»2. ' " 

8.4.2.2 One eighth sample luma interpolation 

The positions labelled *A* in Figure 8-7 represent reference picture samples in integer positions. Other symbols represent 
interpolated values at fractional sample positions. *y">»uis represent 
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Figure 8-7 - Integer samples (<A<) and fractional sample locations for one eighth sample luma interpolation 



The samples at half and quarter sampje positions shall be obtained by applying 8-tap filters with following coefficients: 

- coeffl for sample values at 1/4 positions: (-3, 12, -37, 229, 71, -21, 6, -1), " 

- coeff2 for sample values at 2/4 positions: (-3, 12, -39, 158, 158, -39, 12, -3), 
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cocfG for sample values at 3/4 positions: (-1, 6, -21, 71, 229, -37, 12, -3). 

The samples at one eighth sample positions are defined as weighted averages of reference picture samples at integer, half 
and quarter sample positions. The process for each position is described below. 

- The samples at half and quarter sample positions denoted as *b h * shall be obtained by First calculating ^ 
intermediate values b t by applying 8-tap filter to the nearest samples 4 A' at integer positions in a 
horizontal direction. For left 'b h \ middle 4 b h ' and right V" in Figure 8-7, coefficients coeffl, coeff2 and 
cocfD are used, respectively. The final value of 4 b h ' shall be obtained using b h = Clipl((fc+128)»8). The 
samples at half and quarter sample positions labelled as V* shall be obtained equivalcntly with the filter 
applied in vertical direction. For upper V\ middle V and bottom V* coefficients coeffl, cocffZ and. 
coclT3 are used, respectively. 

- The samples at half and quarter sample positions labelled as 'c" 1 ' and *c q ' shall be obtained by 8-lap 
filtering of the closest intermediate values b in either horizontal or vertical direction to obtain value c\ and 
then the final result shall be obtained using c m =Clip I ((c+32768)» 1 6) or c q = Clip I ((c+32768)» 1 6). 
Filtering in horizontal and vertical direction gives identical results. When filtering in horizontal direction is 
applied? for left 4 c q \ middle 4 c 4 ' and right 4 c q \ coefficients coeffl, coeff2 and^ coefB are used, 
respectively. When filtering in vertical direction is applied, for upper 4 c q \ middle 4 c q * and bottom V p 
coefficients coeffl , cocff2 and coeff3 are used, respectively. For 4 c m ' coefficients coeff2 are used. 

- The samples at one eighth sample positions labelled as W shall be obtained by averaging with truncation 
of the two closest samples at half and quarter sample positions using d = (A+b h )»l, d = (b +b d = 
( A+b v )» I , d = (b h +c q )» 1 , d = (b v +c q )» I , d = <c q +c q )» I , d = (b v +b v )» 1 , or d = (c q +c M )» 1 

- The samples at one eighth sample positions labelled as V shall be obtained by averaging with truncation 
the closest % b h ' and *b v ' samples in diagonal direction using c = (b ,l +b v )»l . 

The samples at one eighth sample positions labelled as 4 g' shall be obtained from the closest integer 
samples *A ? and the 4 c m ' samples using g = (A+3c m +2)»2. 

The samples at one eighth sample positions labelled as and 4 f Vt shall be calculated as I* = (3b"+ 
b v +2)»2and f = (3b v + b h +2)»2. 




Figure 8-8 - Diagonal interpolation for one eighth sample luma interpolation 



8.4.2.3 Chroma interpolation 

Motion compensated prediction fractional chroma samples shall be obtained using Equation 8-13. 

v = {{s ^ d ^ {s ^dnA + d x (s-dnB^(s-dnd y C^d'd y D^s 2 /2)/s 2 (8-13) 



where A, B, C and D are the integer position reference picture samples surrounding the fractional sample location; d K and 
<P are the fractional parts of the sample position in units of one eighth samples for quarter sample interpolation or one 
sixteenth samples for one eighth sample interpolation; and s is 8 for quarter sample interpolation and is 16 for one eighth 
sample interpolation. The relationships between the variables in Equation 8-13 and reference picture positions are 
illustrated in Figure 8-9. 
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Figure 8-9 - Fractional sample position dependent variables in chroma interpolation and surrounding integer 

position samples A, B, C, and D. 



8.5 Intra Prediction 

Two Intra coding modes for macroblocks are described below. Sample values used in the prediction process for intra 
sample prediction shall be sample values prior to alteration by any deblocking filter operations 

8 51 Intra Prediction for 4x4 luma block in Intra_4x4 macroblock type 

Figure 8-10 illustrates the Intra prediction for a 4x4 block. The samples of a 4x4 block containing samples a to p in 
Figure 8-10 are predicted using samples A to Q in Figure 8-10 from neighbouring blocks. Samples A to Q may already 
cTrcumsfantes "** predk,i ° n Any sample A "Q sha » *> e considered not available under the following 

- if they are outside the picture or outside the current slice, 

- if they belong to a macroblock that is subsequent to the current macroblock in raster scan order, 
if they are sent later than the current 4x4 block in the order shown in Figure 6-5, or 

if they are in a non-intra macroblock and constrained_intra_prcd is I 

When samples E-H are not available, the sample value of D is substituted for samples E-H. When samples M-P are not 
available, the sample value of L is substituted for samples M-P. 
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Figure 8-10 - Identification of samples used for intra spatial prediction 



For the luma signal, there are nine intra prediction modes labelled 0, 1, 3, 4, 5, 6, 7, and 8. Mode 2 is 'DC-prediction' 
(see below). The other modes represent directions of predictions as indicated in Figure 8-11. 
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Figure 8-11 - Intra prediction directions 



If adaptive J>Iock_sizcJransfomi_flag == L the intra prediction modes may be used for 4x8, 8x4, and 8x8 blocks as 
specified in subclause 1 2.4. 1. 

8.5.1.1 Mode 0: vertical Prediction 

This mode shall be used only if A, B, C, D are available. The prediction in this mode shall be as follows: 

- a, u i ? m are predicted by A. \ 
h, f, j, n are predicted by B, 

c, g, k, o are predicted by C, 

d, h, 1, p are predicted by D. 

8.5.1.2 Mode 1: horizontal prediction 

This mode shall be used only if 1, J, K, L are available. The prediction in this mode shall be as follows: 
a, b, c, d are predicted by I, 

- e, f, g, h arc predicted by J, 

- i, j, k, 1 arc predicted by K, 
m, n, o, p are predicted by L. 

8.5.1.3 Mode 2: DC prediction 

If all samples A, B, C D, I, J, K, L, are available, all samples arc predicted by (A+B+C+D+I+J+K+L+4)»3. If A, B, C, 
and D are not available and T, J, K, and L are available, all samples shall be predicted by <i+J+K+L+2)»2. If I, J, K, 
and L are not available and A, B, C and D are available, all samples shall be predicted by (A+B+C+D+2)»2. If all 
eight samples are not available, the prediction for all luma samples in the 4x4 block shall be 128. A block may therefore 
always be predicted in this mode. 

8.5.1.4 Mode 3: diagonal down/left prediction 

This mode shall be used only if all A, B, C, D, I, J, K, L, Q are available. This is a 'diagonal 1 prediction. The prediction 
in this mode shall be as follows: 

(A + 2B + C + I + 2J + K + 4) » 3 
(B + 2C + D + J + 2K + L + 4) » 3 
(C + 2D + E + K + 2L + M + 4) » 3 
(D + 2E + F + L + 2M + N + 4) » 3 
(E + 2F + G + M + 2N + O + 4) » 3 
(F + 2G + H + N + 20 + P + 4) » 3 
(G + H + 0 + P + 2)»2 



a is predicted by 

b, e are predicted by 

c, f, i are predicted by 

d, g, j, m are predicted by 
h, k, n are predicted by 
1, o arc predicted by 
p is predicted by 
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8.5.1.5 Mode 4: diagonal down/right prediction 

This mode shall be used only if all A, B, C, D, I, J, K, L, Q are available. This is a 'diagonal' prediction. The prediction 
jn this mode shall be as follows: F 

- m is predicted by: (J + 2K + L + 2) » 2 
i, n are predicted by (I + 2J + K + 2) » 2 
e, j, o are predicted by (Q + 21 + J + 2) » 2 
a, f, k, p are predicted by (A + 2Q + I + 2) » 2 

- b, g, 1 are predicted by (Q + 2A + B + 2) » 2 
c, h are predicted by (A + 2B + C + 2) » 2 

- d is predicted by (B + 2C + D + 2) » 2 
8.5.1.6 Mode 5: vertical-left prediction 

This mode shall be used only if all A, B, C, D, I, J, K, L, Q are inside the slice. This is a 'diagonal' prediction. 

a, j are predicted by (Q + A + I ) » I 

b, k are predicted by (A + B + 1) » I 

c, 1 are predicted by (B + C + 1) » I 
d is predicted by (C + D + 1 ) » I 

- e, n are predicted by (I + 2Q + A + 2) » 2 

- f, o are predicted by (Q + 2A + B + 2) » 2 

- ' g, p are predicted by (A + 2B + C + 2) » 2 

- h is predicted by (B + 2C + D + 2) » 2 

- i is predicted by (Q + 21 + J + 2) » 2 

- m is predicted by (I -f 2J + K + 2) » 2 
8.5.1.7 Mode 6: horizontal-down prediction 

This mode .shall be used only if all A, B, C. D. I, J, K, L, Q are available. This is a 'diagonal' prediction. The prediction 
in this mode shall be as follows: 

a, g are predicted by (Q + 1 + I ) » I 

b, h are predicted by (I + 2Q + A+ 2) » 2 

- c is predicted by (Q + 2A + B+ 2) » 2 

- d is predicted by (A + 2B + C+ 2) » 2 

- e, k are predicted by (I + J + 1 ) » I 

- f, 1 are predicted by (Q + 21 + J+ 2) » 2 
i, o are predicted by (J + K + 1 ) » I 

- j, p are predicted by - (I + 2J + K+2)»2 

- m is predicted by (K + L + I) » 1 

~ n is predicted by (J + 2K + L + 2) » 2 
8.5.1.8 Mode 7: vertical-right prediction 

This mode shall be used only if all A, B, C, D, I, J, K. L, Q are available. This is a diagonal" prediction. The prediction 
in this mode shall be as follows: 

- a is predicted by (2A + 2B + J + 2K + L + 4) » 3 

b, i are predicted by (B + C + I ) » 1 

c, j are predicted by (C + D + 1) » 1 

d, k are predicted by (D + E + 1 ) » I 
1 is predicted by (E + F + 1 ) » I 

- e is predicted by (A + 2B + C + K + 2L + M + 4) » 3 
f , m are predicted by (B + 2C + D + 2) » 2 
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- g, n arc predicted by (C + 2D + E + 2) » 2 

- h ? o are predicted by (D + 2E + F + 2) » 2 

- p is predicted by (E + 2F + G + 2) » 2 

8.5.1.9 Mode 8: horizontal -up prediction 

This mode shall be used only if all A, B, C, D, I, J, K, L, Q are available. This is a 'diagonal' prediction. The prediction 

in this mode shall be as follows: 

- a is predicted by (B + 2C + D + 21 + 2J + 4) » 3 

- b is predicted by (C + 2D + E + I + 2J + K + 4) » 3 

- c\ e are predicted by (J + K + I ) » I 

- d, f are predicted by (J + 2K + L + 2) » 2 

- g, i are predicted by (K + L + I ) » 1 

- h, j are predicted by (K + 2L + M + 2) » 2 

- I, n are predicted by (L + 2M + N + 2) » 2 

- k, m are predicted by (L + M + 1 ) » I 

- o is predicted by (M + N + 1) » I 

- p is predicted by (M + 2N + O + 2) » 2 

8.5.2 Intra prediction for luma block in Intra_16xl6 macroblock type 

Denote the block lo be predicted as having sample locations 0 to 15 horizontally and 0 to 15 vertically. The notation 
P(x,y) is used, where x = 0..I5 corresponds to horizontal positions and y = 0..I5 corresponds to vertical positions. P(x,- 
I), x=<)..15 are the neighbouring samples above the block and P(-I,y), y=O .I5 are the neighbouring samples to the left of 
the block. Prcd(x,y) *x,y = 0..15 is the prediction for ihc luma macroblock samples. There arc 4 different prediction 
modes as specified in subclauses 8.5.2.1 to 8.5.2.4. 

Samples P(x,- 1 ) or P(-l ,y) shall be considered not available under the following circumstances: 
if they are outside the picture or outside the current slice, or 
if they are in a non-intra macroblock and constraincd_inlra_pred is I. 

8.5.2.1 Mode 0: vertical prediction 

This mode shall be used only if all neighbouring samples P(x, -1) are available. 

Pred(x, y) = P(x, -1), x, y=O..I5 (8-14) 

8.5.2.2 Mode 1: horizontal prediction 

This mode shall be used only if all neighbouring samples P(- 1, y) are available. 

Pred(x, y) = P(- 1 , y), x, y=0.. 15 (8-1 5) 

8.5.23 Mode 2: DC prediction 

Pred(x, y) = -1) + /) + 16] » 5 x, y=0.. 15 (8-16) 

If the neighbouring samples P(x, - 1) are not available and the neighbouring samples P(- 1, y) are available, the prediction 
for all luma samples in the macroblock is given by Equation 8-17. 

Pred(x,y)= [£P(-l./) + 8]>>4 x,y=0..15, (8-17) 

If the neighbouring samples P(-l f y) arc not available and the neighbouring samples P(x, -1) are available, the prediction 
for all luma samples in the macroblock is given by Equation 8-18. 
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15 

Pred(x,y) = [£P(jtV-l) + 8]>> 4 x,y=0..15, 

x=0 V ' 



If none of the neighbouring samples P(x,-1) and P(-l,y) are available, the prediction for all luma samples in the 
macroblock shall be 1 28. K 

8.5,2.4 Mode 3: plane prediction 

This mode shall be used only if all neighbouring samples P(x, -1) and P(-l, y) are available. 

Pred(x,y) = Clipl( (a + b (x-7) + C .(y-7) +16) » 5 ), (8 . 19) 

where: 



a = 16 (P(-1,15) + P(15,-l)) (g 2Q) 

b = (5*H+32)»6 

c = (5*V+32)»6 
and H and V are defined in Equations 8-23 and 8-24. 

8 

H = (P(7 + x-\) - P(l - x-l)) 



(8-21) 
(8-22) 



(8-23) 



(8-24) 



v = £ y • + y) - P(-iJ - y)) 

8.5.3 Prediction in intra coding of chroma blocks 

The chroma in intra macroblocks is predicted in a manner very similar to the luma block in Intra_16xl6 macroblock type 
(subclause 8.5.2), using one of four prediction modes. The same prediction mode is applied to both chroma blocks but it 
is independent of the prediction mode used for the luma. ' 

NOTE - If any portion of the luma macroblock is coded in intra mode, the entire chroma macroblock is coded intra. 

Let P(x,-1), x=0..7 be the neighbouring samples above the chroma macroblock and P(-l,y), y=0..7 be the neighbouring 
samples to the left of the chroma macroblock. Pred(x,y), x,y = 0..7 is the prediction for the whole chroma macroblock 
and is computed as follows for the four prediction modes. Samples P(x,-1) or P(-l,y) shall be considered not available' 
under the following circumstances: 

if they are outside the picture or outside the current slice, 

if they are in a non-intra macroblock and constrained_intra_pred is 1 . 
Whenever P(x,y) is not available, P(x,y) is inferred to have the value 128 except as specified in subclause 8.5.3.3. 
edges^ horizontal and vertical prediction, P(x,y) is first filtered using a { 1,2,1 }/4 filter, with pixel replication at the 

8.5.3.1 Mode 0: vertical prediction 

F(0.-1) = (PfO.-l) + P(I,-1) + 1) » 1 (8 _ 25 ) 

F(x-l) = (P (x - 1,-1) + 2 x P(jc,-1) + P( X +1,-1) + 2) » 2 , x= 1 ,. . . ,6 (8-26) 
F(7,-l) = (P(6,-l) + P(7,-l) + 1) » 1 (8 _ 27 ) 
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Pred(x, y) = F(x, -1), x, y=0..7 



8.5 3.2 Mode 1: horizontal prediction 

F(-1,0) - (P(-1,0) + P(-U) + 1) » 1 

F(-l, y) = (P(-l, y-l) + 2xP(-l, y) + P(-l. y + 1) + 2) » 2, y=l,—,6 
F(-|,7) = (F(-l,6) + P(-l,7) + 1) » I 
Pred(x, y) = F(-l, y), x, y=0..7 
8.5.3 .3 Mode 2: DC prediction 

If all samples P(-l,n) and P(n,-1) used in Equation 8-33 are available, the prediction is formed 
Pred(x, y) = |£( P( _,,,o + n«,-l)))+8j>>4 x,y=0..7, 

If the 8 samples P(-l,n) are not available, the prediction is formed as 

»3 x,y=0..7, 



Pred(x.y) = j^g P(n ~ !) j + 4 ] : 



If the 8 samples P(n,-1) are not available, the prediction is formed as 



Pred(x,y) = 



I§'H +4 ] 



» 3 x,y=0..7, 



If all 16 samples are not available, the prediction Pred(x,y) for all samples x,y=0..7is 128. 

8.5.3.4 Mode 3: plane prediction 

For the plane mode, the prediction is formed as: 

Pred(ij) = Clipl( (a + b (i-3) + c-(j-3) +16) » 5 ), ij=0,...,7 

where: 

a= 16-(P(-1,7) + P(7,-1)) 

b = (17*H+16)»5) 

c = (17*V+16)»5) 
and H and V are defined as: 

H = (A3+ i-D - PQ ~ «'.-D) 
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V = £ j ■ W-1,3 + /) - />(-U - ;)) (8-39) 



8.6 Transform coefficient decoding and picture construction prior to deblocking 

This subclause defines aspects related to transform coefficient decoding. 
8.6.1 Zig-zag scan 

The decoder maps the sequence of transform coefficient levels to the transform coefficient level positions For this 
mapping, the scanning pattern is shown in Figure 8-12. F"=>»'"ns. ror mis 




In the case of 16x16 intra macroblocks, the coefficients of the 4x4 luma DC transform are scanned in the same scan 
order as ordinary 4x4 coefficient blocks. Then for each 4x4 block of luma coefficients with AC coefficient to s^n 'he 
1 5 remaining coefficients are scanned by starting the zig-zag scan at its second position. 

The coefficients of the 2x2 chroma DC transform are scanned in raster order. Then for each 4x4 block of chroma 
s^d potion COefficients to scan - * e 15 remai "i«g coefficients are scanned by starting the zigSg scan aUts 

If adaptive block_size_transform flag = = 1, 4x4, 4x8, 8x4, and 8x8 luma coefficient blocks are scanned using zig-zag 
scans and field scans as specified in subclause 12.4.2. 6 B g 

8.6.2 Scaling and transformation 

There are 52 different values of QP values that are used, ranging from 0 to 51, inclusive. The value of QP C for chroma is 
SZETfe CUrrent V , a L UC lS P ^ The SC3,ing equa,i ° nS m defined such that the equivalent scaling parameter 

QP S 1^x1^ ' nCremem " Q ' * " SCa " ng of approximately 12% from one 

The value of QP C shall be determined from the value of QP Y as specified in Table 8-2: 

Table 8-2 - Specification of QP C as a function of QP Y 
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QP Y shall be used as the QP to be applied for luma scaling and QP C shall be used for chroma scaling. 
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The coefficients R~ m) defined in Equation 8-40 are used in Equations 8-43, 8-44, 8-46, 8-47 and 8-48. 



V-o for {(0,0), (0,2), (2,0), (2,2)}, 

V ml for (i,j)e {(U), (1,3), (3,1), (3,3)}, 
V m2 otherwise; 



(8-40) 



where the first and second subscripts of V are row and column indices, respectively, of the matrix defined as: 



V = 



10 


16 


13 


11 


18 


14 


13 


20 


16 


14 


23 


18 


16 


25 


20 


18 


29 


23 



(8-41) 



8.6.2.1 Lama DC coefficients in Intra 16x16 macroblock 

After decoding the coefficient levels for a 4x4 block of luma DC coefficients coded in 16x16 intra mode and assembling 
these into a 4x4 matrix C of elements c iJf a transform process shall be applied in a manner mathematically equivalent to 
the following process. The process uses application of a transform before the scaling process. 

The transform for the 4x4 luma DC coefficients in 16x16 intra macroblocks is defined by: 



1111 
1 1-1—1 
1-1-1 1 
1-1 1-1 



-10 



-20 



^01 



-21 



-30 C 31 



-02 



-12 



-22 



^32 



u 03 



-13 



-23 



u 33 J 



1111 
1 1-1-1 
1-1-1 1 
1-1 1-1 



(8-42) 



A bitstream conforming to this Recommendation | International Standard shall not contain indicated data that results in 
any element of F that exceeds the range of integer values from -2 15 to 2 l5 -l, inclusive. 

After the transform, scaling is performed according to the following: 

a) If QP is greater than or equal to 12, then the scaled result shall be calculated as 



DC;j = [F # - Rg P%6) }«(QP/6- 2), i, j = 0,. . . ,3 . 

b) If QP is less than 12, then the scaled results shall be calculated as 

DC, = [F 9 • Rg™ + 2 X - QP ' 6 ] » (2 - QP/ 6), i, 7=0 3 



(8-43) 



(8-44) 



A bitstream conforming to this Recommendation | International Standard shall not contain data that results in element of 
DQj that exceeds the range of integer values from -2 15 to 2 15 -1 , inclusive. 

8.6.2.2 Chroma DC coefficients 

After decoding the coefficient levels for a 2x2 block of chroma DC coefficients and assembling these into a 2x2 matrix C 
of elements the transform process is applied before the scaling process. 

Definition of transform: 



(8-45) 
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A bitstream conforming to this Recommendation | International Standard shall not contain indicated data that results in 
any element of F that exceeds the range of integer values from -2 ,s to 2 IS -1, inclusive. 

After the transform, scaling is performed according to the following. 

a) If QP is greater than or equal to 6, then the scaling result shall be calculated as 

^=[F r C a6) ]«(e/ > /6-l), W = 0,...,3. (8 _4 6) 

b) If QP is less than 6, then the scaling results shall be calculated by 

*>C 9 =lF 9 .R«r»]»l, U = 0,...,3. 

A bitstream conforming to this Recommendation | International Standard shall not contain indicated data that results in 
any element of DCy that exceeds the range of integer values from -2 1S to 2 ,5 -I, inclusive. data that reSU,ts ,n 

8.6.23 Residual 4x4 blocks 

Scaling of 4x4 block coefficient levels c 0 other than those as specified in subclauses 8 62 1 and 8 62 2 shall ho 
performed according to Equation 8-48 o.u.^.i ana n.o.z.z shall be 

w 9 =Ky ^r* 6) 3«(2/ > /6), i,j = 0,...,3. (8 _4 8) 

A bitstream conforming to this Recommendation | International Standard shall not contain indicated data that result, in » 
value of wy that exceeds the range of integer values from -2 15 to 2 ,s -l , inclusive. naicatea <»ata that results m a 

iSLS^SZT 9 blOCk ° f SC3,ed tranSfOTm COeffiCientS «™*** — • -trix W of 



w = 



w oo w 01 w 02 m> 03 

w, 0 w n m>, 2 iv I3 

W 20 W 2| VV 23 

^30 ^31 W 32 W 33 



(8-49) 



in which the wo, element may be a result DC 0 from Equation 8-43, 8-44, 8-46, or 8-47; or may be from Eauation 8-48 a , 
appropriate, the transform process shall convert the block of reconstructed u-ansform «S£2£? TblocJ : of^utout 
samples in a manner mathematically equivalent to the following process: oerncients to a block of output 

a) First, each row of reconstructed transform coefficients is transformed using a one-dimensional transform, 

b) Second, each column of the resulting matrix is transformed using the same one-dimensional transform. 
The one-dimensional transform is defined as follows for four input samples w 0 , w„ w 2 , w 3 . 

a) First, a set of intermediate values is computed: 



Zo=w Q + w 2 
Zt = wo - w 2 

Z2=(W, » 1)-W 3 
Z 3 = Wj + (w 3 » 1) 

b) Then the transformed result is computed from these intermediate values 

Xo = Zo + Z3 



(8-50) 
(8-51) 
(8-52) 
(8-53) 

(8-54) 
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X\ = Z! + Z2 



*2 = Zl - *2 



(8-55) 
(8-56) 



*3 = Zo " 23 



(8-57) 



A bitstream conforming to this Recommendation | International Standard shall not contain indicated data that results in a 
value of zo, Zu ^ *o* x * or x 3 that exceeds the range of integer values from -2 to 2 -1, inclusive, in either the 
first (horizontal) or second (vertical) stage of application of this transformation process. A bitstream conforming to this 
Recommendation | International Standard shall not contain indicated data that results in a value of x u x 2 > or jc 3 that 
exceeds the range of integer values from -2 15 to 2 l5 -33, inclusive, in the second (vertical) stage of application of this 
transformation process- 
After performing the transform in both the horizontal and vertical directions to produce a block of transformed samples, 







X 02 


*03 


10 




X I2 


X 13 


X 20 




X 22 


X'n 


X 30 




X 32 


*33_ 



(8-58) 



the final reconstructed sample residual values shall be obtained as 

i V 

X" =[z' f +2 s ]»6 



(8-59) 



If adaptive_block_size_transform_flag = = 1, scaling and inverse transform for 4x8, 8x4, and 8x8 coefficient blocks is 
specified in subclause 12.4.3. 

8.63 Adding decoded samples to prediction with clipping 

Finally, the reconstructed sample residual values X" from Equation 8-59 are added to the prediction values P tj from 
motion* compensated prediction or spatial prediction and clipped to the range of 0 to 255 to form the final decoded 
sample result prior to application of the deblocking filter: 

S^^ClipKP.y + X'-) (8- 6 °) 
8.7 Deblocking Filter 

A conditional filtering shall be applied to all macroblocks of a picture. This filtering is done on a macrobiock basis, with 
macroblocks being processed in raster-scan order throughout the picture. For luma, as the first step, the 16 samples of the 
4 vertical edges of the 4x4 raster shall be filtered Jjeginning with the left edge, as shown in Figure 8-13. Filtering of the 4 
horizontal edges (vertical filtering) follows in the same manner, beginning with the top edge. The same ordering applies 
for chroma filtering, with the exception that 2 edges of 8 samples each are filtered in each direction. This process also 
affects the boundaries of the already reconstructed macroblocks above and to the left of the current macrobiock. Picture 
edges are not filtered. 

When mb_adaptive_frame_field_flag = 1, a MB may be coded in frame or field decoding mode. For frame MB, 
deblocking is performed on the frame samples. In this case, if neighbouring MB pairs are field MBs, they shall be 
converted into frame MB pairs (Figure 8-3) before deblocking. For field MB, deblocking is performed on the field 
samples of the same field parity. In this case, if neighbouring MB pairs are frame MBs, they shall be converted into field 
MB pairs (Figure 8-3) before deblocking. 



76 



ITU-T Rec H.264 (2002 E) 



3D: <XP_ 



_1 100641A l_> 



ISO/IEC FCD 14496-10 



16*16 Macroblock 



16*16 Macroblock 
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Horizontal edges 
(luma) 

Horizontal edges 
(chroma) 



Vertical edges Vertical edges 
(luma) (chroma) 



Figure 8-13 - Boundaries in a macroblock to be filtered (luma boundaries shown with solid lines and chroma 

boundaries shown with dotted lines) 



Intra prediction of a macroblock shall be done using the unfiltered content of the already decoded neighbouring 
macroblocks. Depending on the implementation, the values necessary for intra prediction may need to be stored before 
filtering in order to be used in the intra prediction of the macroblocks to the right and below the current macroblock. 

When pic_structure indicates a field picture all decoding operations for the deblocking filter are based solely on samples 
within the current field. ^ 

8.7.1 Content dependent boundary filtering strength 

For each boundary between neighbouring 4x4 luma blocks, a "Boundary Strength" Bs is assigned as shown in Figure 8- 
14. If Bs=0, filtering is skipped for that particular edge. In all other cases filtering is dependent on the local sample 
properties and the value of Bs for this particular boundary segment. 

For each edge, if one of the neighbouring blocks is intra-coded, a relatively strong filtering (Bs=3) is applied. A special 
procedure with even stronger filtering might be applied on intra-coded macroblock boundaries (Bs=4). If neither of the 
blocks are intra-coded and at least one of them contains non-zero coefficients, medium filtering strength (Bs=2) is used 
If none of the previous conditions are satisfied, filtering takes place with Bs=l if at least one of the following conditions 
is satisfied: (a) prediction of the two blocks is formed using different reference frames or a different number of reference 
frames, (b) a pair of motion vectors from the two blocks is referencing the same frame and either component of this pair 
has a difference of more than one sample. Otherwise filtering is skipped for that particular edge (Bs=0). 
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Figure 8-14 - Flow chart for determining the boundary strength (Bs), for the block boundary between two 
neighbouring blocks p and q, where V,(p,x), V,(p,y) and V 2 (p, x), V 2 (p, y) are the horizontal and vertical 
components of the motion vectors of block p for the first and second reference frames or fields. 



8.7.2 



Thresholds for each block boundary 



P3 


P2 


Pi 


pO| qO 


q1 


q2 


q3 



Figure 8-15 - Convention for describing samples across a 4x4 block horizontal or vertical boundary 



In the following description, the set of eight samples across a 4x4 block horizontal or vertical boundary is denoted as 
shown in Figure 8-15 with the actual boundary lying between po and q 0 . Uppercase letters indicate filtered samples and 
lower case letters indicate unfiltered samples with regard to the current edge filtering operation. However, p! and pi may 
indicate samples that have been modified by the filtering of a previous block edge. 

Sets of samples across this edge are only filtered if the condition 

Bs*0 && \p 0 -qo\<a \pi-Po\<P foi-qo\<P (8-61) 



is true. The values of the thresholds a and p are dependent on the average value of QP for the two blocks as well as on a 
pair of index offsets t THter_Offset_A" and "Filter_OffsetJB" that may be transmitted in the slice header for the purpose 
of modifying the charact ristics of the filter. The average QP value for the two blocks is computed as QP av = 
(QP P +QPq)»l- The index used to access the a-table (Table 8-3), as well as the CO-table (Table 8-4) that is used in the 
default filter mode, is computed as: 
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Index A = Clip3(0, 51, QP av + Filter_Offset_A) 
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(8-62) 



NOTE - In SP and SI slices, QP av is calculated in the same way as in other slice types. QS Y from Equation 7-8 is not used in the 
deblocking filter. 

The index used to access the p-table (Table 8-3) is computed as: 

Index B = Clip3(0, 51, QP av + Filter_Offset JB) (8-63) 

If adaptive_block_size_transform_flag = = 1, Index A and Index B are calculated as specified in subclause 12.4.4. 
The relationships between the indices (Equations 8-62 and 8-63) and the thresholds (a and 0) are shown in Table 8-3. 

Table 8-3 - QP. V and offset dependent threshold parameters a and P 

Index A (for a) or Index B (for p) I 





0 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


22 


23 


24 


25 


a 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4 


4 


5 


6 


7 


8 


9 


10 


12 


13 


p 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


2 


2 


2 


3 


3 


3 


3 


4 


4 


4 






















Table 8-3 (concluded) 

Index A (for a) or Index D (for p) 






















26 


27 


28 


29 


30 


31 


32 


33 


34 


35 


36 


37 


38 


39 


40 


41 


42 


43 


44 


45 


46 


47 


48 


49 


50 


51 


a 


15 


17 


20 


22 


25 


28 


32 


36 


40 


45 


50 


56 


63 


71 


80 


90 


101 


113 


127 


144 


162 


182 


203 


226 


255 


255 


P 


6 


6 


7 


7 


8 


8 


9 


9 


10 


10 


11 


11 


12 


12 


13 


13 


14 


14 


15 


15 


16 


16 


17 


17 


18 


18 



8.7.3 



Filtering of edges with Bs < 4 



Two types of filtering are defined. In the default case with 0 < B < 4, Equations 8-64, 8-65 and 8-66 are used to filter on 
and q 0 : K 



A = Clip3( -C, C, (((qo-po) «2 + (p,- q,) + 4) » 3) ) 
Po = Ctip\(p 0 +A) 
Qo = C\\p\(q 0 - A) 



(8-64) 
(8-65) 
(8-66) 



where C is determined as specified below. 
The two intermediate threshold variables 

<*p= \P2~Po\ 
<*q = l?2 - qo\ 



(8-67) 
(8-68) 



shall be used to determine whether filtering for the luma samples p, and q, is taking place at this position of the edge. 
lfa p <P for a luma edge, a filtered sample P, shall be produced as specified by 

P, = P, + Clip3( -CO, CO, (P2 + (p 0 + q 0 )»l-(pj«l)) » 1 ) (8-69) 



If a, < /?for a luma edge, a filtered sample Q, shall be produced as specified by 



.) 100641 A_l_> 



ITU-T Rec. H.264 (2002 E) 



79 



ISO/IEC FCD 14496-10 

Qj = + Clip3( -CO, Cft (? 2 + ( Po + qo )»1 - (<?/«!) » 1) 

where CO is specified in Table 8-4. Chroma samples pi and q, are never filtered. 

C is determined by setting it equal to CO and then incrementing it by one if a p </3, and again by one if a q <fi. 

Table 8-4 - Value of filter clipping parameter CO as a functi n of Index A and Bs 



(8-70) 





lndex A 




0 


1 


■2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


22 


23 


24 


25 


Bs= 1 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


1 


1 


Bs = 2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


1 


1 


1 




Bs = 3 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


1 


I 


1 


1 


1 




1 


1 


1 








Table 8-4 (concluded) 




Index A 




26 


27 


28 


29 


30 


31 


32 


33 


34 


35 


36 


37 


38 


39 


40 


41 


42 


43 


44 


45 


46 


47 


48 


49 


50 


51 


Bs=l 


1 


1 


1 


1 


1 


1 


1 


2 


2 


2 


2 


3 


3 


3 


4 


4 


4 


5 


6 


6 


7 


8 


9 


10 


11 


13 


Bs = 2 


1 


1 

_t _ 


1 


1 




2 


2 


2 


2 
. i_ _ 


3 


3 


3 


4 


4 


5 


5 


6 


7 


8 


8 


10 


11 


12 


13 


15 


17 


Bs = 3 


1 


2 


2 


2 


2 


3 


3 


3 


4 


4 


4 


5 


6 


6 


7 


8 


9 


10 


11 


13 


14 


16 


18 


20 


23 


25 



8.7.4 Filtering of edges with Bs = 4 

When Bs is equal to 4 and the following condition holds: 

a p <fi && \p 0 -qo\ < ((a» 2) + 2) 

filtering of the left/upper side of the block edge is specified by Equations 8-72 and 8-73. 
Po=(P2 + 2*p, + 2*p 0 + 2*q 0 + q,+4)»3 

Pj = (P2 + Pi+Po + 4o+2)»2 

In the case of luma filtering, the filter in Equation 8-74 is also applied. 

P 2 = ( 2*p 3 + 3*p 2 + p,+Po + qo + 4)»3 

Otherwise, if the condition of 8-71 does not hold, the filter in Equation 8-75 is applied. 
P 0 = ( 2*pi + po + qi + 2) » 2 

Similarly, for filtering of the right/lower side of the edge, if the following condition holds: 
a q < p && \p 0 - ?o| <((a» 2) +2) 

filtering is defined by Equations 8-77 and 8-78. 

Qo = ( Pj + 2*p 0 + 2*q 0 + 2*q, + q 2 + 4)»3 



(8-71) 

(8-72) 
(8-73) 

(8-74) 

(8-75) 

(8-76) 

(8-77) 
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Qi = ( Po + 9o + qj + #2 + 2) » 2 (8-78) 
In the case of luma filtering, the filter in Equation 8-79 is also applied. 

Q 2 = ( 2*4j + 3*<? 2 + 4, + q 0 + Po + 4) » 3 (8-79) 



Otherwise, if the condition of 8-76 does not hold, the filter in Equation 8-80 is applied: 
Qo = (2*q 1 + q 0 + P! +2)»2 



(8-80) 



Entropy Coding 



9.1 Variable Length Coding 
9AA Exp-Golomb entropy coding 

The table of Exp-Golomb codewords is written in the following compressed form. 
1 

0 1 x0 
0 0 1 xl xO 
0 0 0 1 x2 xl xO 
0 0 0 0 1 x3 x2 xl xO 



where xn take values 0 or 1. A codeword can be referred by its length in bits (L = 2n-H and TNFO - *n V 1 v n M 

ll^ZT of bhs in INFO is - 1 bits - The codewords « — ^ ?-S upw^T 'SE 

Code.num = 2^ + INFO -1 (172 denotes division with truncation and INFO = 0 when L = 1) The first 10 code 

ztzst t2r sp exp,icit,y in Tab,e 9 - L As an examp,e - for the code number 5 ' L = ^ = 

Table 9-1 - Code number and Exp-Golomb codewords in explicit form and used as ue(v) 



Code_num 


Code word 


0 


1 


1 


0 10 


2 


0 11 


3 


0 0 10 0 


4 


0 0 10 1 


5 


0 0 110 


6 


0 0 111 


7 


0 0 0 1 0 0 0 


8 


0 0 0 1 0 0 1 


' 9 


0 0 0 1 0 1 0 



When L (L = 2N-1) and INFO is known, the regular structure of the table makes it possible to create a codeword usnin* 
the structure of the table. A decoder shall decode a codeword by reading in N bit prefix foHowS by N-f SSS TS 
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INFO are then available. For each parameter to be coded, there is a conversion rule from the parameter value to the code 
number (or L and INFO). 

9.1.2 Unsigned Exp-Golomb entropy coding 

The value of syntax lements that are represented by unsigned Exp-Golomb entropy coding directly corresponds to the 
code_num value of Table 9-1. This type of entropy coding is indicated via ue(v). 

9.1.3 Signed Exp-G tomb ntropy coding 

The syntax elements that are represented by signed Exp-Golomb entropy coding are assigned to the code.num by 
ordering using their absolute values in increasing order and representing the positive value with the lower code_num. 
Table 9-2 provides the assignment rule. 

Table 9-2 - Assignment of symbol values and code_nums for signed Exp-Golomb entropy coding se(v) 



Code number 


Symbol value 


0 


0 


1 


1 


2 




3 


2 


4 


-2 


5 


3 


6 


-3 


7 


4 


8 


-4 


9 


5 


10 


-5 


k 


(-l) k+l Ceil(k-r2) 



This type of entropy coding is denoted as se(v). 

9.1.4 Mapped Exp-Golomb entropy coding 

Table 9-3 specifies the assignment of all mapped Exp-Golomb-coded slice data symbols. This type of entropy coding is 
indicated via me(v). These symbols are decoded differently when entropy_coding_mode = = 1. 

If adaptive J>lock_size_transfonn_ilag == 1. additional syntax elements are mapped to the Exp-Golomb code as 
specified in Table 12-5. 

Table 9-3 - Assignment of codeword number and parameter values for mapped Exp-Golomb-coded symbols 



Code 
number 


coded_block ^pattern 

assignment to 
macroblock prediction 
types 


Tcoef f_chroma_DC 1 


Tcoeff_chroma_AC 1 
Tcoeffjumal 
Zig-zag scan 




Intra, S Intra 


Pred, SPred 


Level 


Run 


Level 


Run 


0 


47 


0 


EOB 




EOB 




1 


31 


16 




0 


I 


0 


2 


15 


1 


-1 


0 


-1 


0 


3 


0 


2 


2 


0 


1 


1 
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41 


32 


27 


6 


1 


2 


5 


42 


33 


29 


-6 


1 


-2 


5 


43 


34 


30 


5 


2 


2 


6 


44 


36 


22 


-5 


2 


-2 


6 


45 


40 


25 


5 


3 


2 


7 


46 


38 


38 


-5 


3 


-2 


7 


' 47 


41 


41 


8 


0 


2 


8 


K 






sec below 


sec below 


see below 


see below 



For the entries above the horizontal line, the table is needed for relation between code number and Le vel/Run/EOB . For 
the remaining Level/Run combination there is a simple rule. The Level/Run combinations are assigned a code number 
according to the following priority: 1 ) sign of Level (+ -) 2) Run (ascending) 3) absolute value of Level (ascending). 

9.1.5 Entropy coding for Intra 

In intra mode, prediction is always used for each sub block in a macroblock. 
9.1.5.1 Coding of Intra 4x4 and SIntra 4x4 prediction modes 

The chosen infra-prediction mode (intra_pred_jnode) of a 4x4 block is highly correlated with the prediction modes of 
adjacent blocks. This is illustrated in Figure 9-1. When the prediction modes of A and B are known (including the case 
that A or B or both are outside the slice) the most probable mode (most_probable_mode) of C is given. If one of the 
blocks A or B is "outside" the most probable* mode is equal to prediction mode 2. Otherwise it is equal to the minimum 
of modes used for blocks A and B. When an adjacent block is coded by 16x16 intra mode, prediction mode is "mode 2: 
DC prediction"; when it is coded a non-intra macroblock, prediction mode is "mode 2: DC prediction" in the usual case 
and "outside" in the case of constrained intra update. 

To signal prediction mode number for a 4x4 block first parameter use_most_probable_mode is transmitted. This 
parameter is represented by 1 bit codeword and can take values 0 or 1. If use_most_probable_mode is equal to 1 the most 
probable mode is used. Otherwise an additional parameter remaining_mode_selector, which can take value from 0 to 7 is 
sent as 3 bit codeword. The codeword is a binary representation of remaining_mode_selector value. The prediction mode 
number is calculated as: 

if (remaining_jnode_selector < most_probable_mode) 
i ntra_pred_mode - remaining_mode_selector; 

else 

intra_pred_mode = remaining_mode_selector+ 1 ; 
The ordering of prediction modes assigned to blocks C is therefore the most probable mode followed by the remaining 
modes in the ascending order. 
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A 
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9 


12 


13 


B 


C 




10 


11 


14 


15 



b 



Figure 9-1 - a) Prediction mode of block C to be established, where A and B are adjacent blocks, b) order of intra 

prediction information in the bitstream 



9.1.5.2 Coding of mode information for Intra-16xl6 mode 

Three numbers are specified at the end of the names of lntra-16x!6 modes as defined in Table 7-10 as a function of 
mb_type. The first of these numbers is termed /mode and ranges from 0 to 3, inclusive. The second is termed nc and 
contains the coded_block_pattem bits for chroma as specified in subclause 9.2.1.6. The third and final of these numbers 
is termed acjlag. acjlag equal to zero indicates that there are no AC coefficients in the 16x16 block, ac flae equal to 
1 indicates that at least one AC coefficient is present for the 16x16 block, requiring scanning of AC coefficient values for 
all 16 of the 4x4 blocks in the 16x16 block. 

9.1.6 Context-based adaptive variable length coding (CAVLC) of transform coefficients 

CAVLC (Context-based Adaptive VLC) is the method used for decoding of transform coefficients. The following 
coding elements are used: 6 

1. If there are non-zero coefficients, it is typically observed that there is a string of coefficients at the highest 
frequencies that are ±1. The coding element coeff.token gives the total number of coefficients (from now 
referred to as TotalCoeffs) and also contains the number of "Trailing Is" (from now referred to as Tls). 

2. For Tls the sign is decoded from trailing_ones_sign and the level magnitude is 1. 

3. For coefficients other than the Tls, level information is decoded from coeffjevel. 

4. The Run information is decoded. Since the number of coefficients is already known, this limits possible values 
for Run. Run is split into the Total number of zeros before all coefficients and Run before each non-zero 
coefficient, given by total_zeros and run_before. 

Zig-zag scanning as described in subclause 9.4.1 is used, but in the decoding of coefficient data, both levels and runs the 
scanning is done in reverse order. Therefore the signs of Tls are decoded first (in reverse order), then the Level 
information of the last coefficient in the zig-zag scan order not included in the Tls, and so on. Run information is 
decoded similarly. First Total number of zeros in Runs is decoded, followed by Run before the last nonzero coefficient 
in the zig-zag scan order, and so on. 

If adaptive_block_size_transform_flag = = 1, the VLC method for decoding 4x4, 4x8, 8x4, and 8x8 luma coefficient 
blocks is specified in subclause 12.5.1 . 

9.1.6.1 Entropy decoding of the number of coefficients and trailing ones: coeff.token 

The syntax element coeff_token is decoded using the VLC specified in Tables 9-4 to 9-7. 

Four VLC tables are used for combined decoding of number of coefficients and Tls, i.e. one codeword signals both 
parameters. VLCs are listed in the tables below. Tls is clipped to 3. Any remaining trailing Is are decoded as normal 
levels. The variable TotalCoeff is the value returned by the function total_coeff( ); the variable Tls is the value returned 
by the function trailmg_ones( ); and the variable NumCoeff related to these quantities by TotalCoeff-Tls. 

Table 9-4 - coeff.token: total_coeff( ) / trailing_ones( ): Num-VLCO 



traiIing_ones( ) 


0 


1 




> " 1 
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total_coeff( ) 










0 


1 


- 


- 




I 


000101 


01 


- 


- 


2 


00000111 


000100 


001 


- 


3 


000000111 


00000110 


0000101 ! 


00011 


4 


0000000111 


0000001 10 


00000101 


000011 


5 


00000000111 


0000000110 


000000101 


0000100 


6 


0000000001111 


00000000110 


0000000101 


00000100 


7 


0000000001011 


0000000001110 


00000000101 


000000100 


8 


0000000001000 


0000000001010 


0000000001 101 


0000000100 


9 


00000000001111 


00000000001110 


0000000001001 


00000000100 


10 


00000000001011 


00000000001010 


00000000001101 


0000000001100 


11 


000000000001 lit 


000000000001110 


00000000001001 


00000000001100 


12 


000000000001011 


000000000001010 


000000000001101 


00000000001000 


13 


0000000000001111 


000000000000001 


000000000001001 


000000000001100 


14 


0000000000001011 


0000000000001110 


0000000000001101 


000000000001000 


15 


0000000000000111 


0000000000001010 


0000000000001001 


0000000000001100 


16 


0000000000000100 


0000000000000110 


0000000000000101 


0000000000001000 



Table 9-5 - coeff_tokeii: total_cocff( ) / trailing_ones( ): Num-VLCl 



traiIing_ones( ) 


0 


1 


2 


3 


total_coeff( ) 










0 


11 










001011 


10 






2 


000111 


00111 


Oil 




3 


0000111 


001010 


001001 


0101 


4 1 


00000111 


000110 


000101 


0100 


5 


00000100 


0000110 


0000101 


00110 


6 


ooooooni 


00000110 


00000101 


001000 


7 


00000001111 


000000110 


000000101 


000100 


8 


00000001011 


00000001110 


00000001101 


0000100 


9 


000000001111 


00000001010 


00000001001 


000000100 


10 


000000001011 


000000001110 


000000001101 


00000001100 


11 


000000001000 


000000001010 


000000001001 


00000001000 


12 


0000000001111 


0000000001110 


0000000001101 


000000001100 


13 


0000000001011 


0000000001 010 


0000000001001 


0000000001100 
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14 


00000000001 1 1 


00000000001011 


0000000000110 


0000000001000 


15 


00000000001001 


00000000001000 


00000000001010 


0000000000001 


16 


000000000001 1 1 


00000000000110 


00000000000101 


00000000000100 



Table 9-6 - coefT_token: totaLcoeff( ) / trail ing_ones( ): Num-VLC2 



trail ing^ones( ) 


0 


1 


2 


3 


total_coeff( ) 










0 


1111 


- 


- 


- 


1 


001111 


1110 


- 


- 


2 


001011 


01111 


1101 


- 


3 


001000 


01100 


oino 


1100 


4 


0001111 


01010 


01011 


1011 


5 


000101 1 


01000 


01001 


1010 


6 


0001001 


001110 


001101 


1001 


7 


0001000 


001010 


001001 


1000 


8 


00001111 


0001110 


0001101 


01101 


9 


00001011 


00001110 


0001010 


001100 


10 


000001111 


00001010 


00001101 


0001100 


11 


000001011 


000001110 


00001001 


00001100 


12 


000001000 


000001010 


000001101 


00001000 


13 


0000001101 


000000111 


000001001 


000001 100 


14 


0000001001 


0000001100 


0000001011 


0000001010 


15 


0000000101 


0000001000 


0000000111 


0000000110 


16 


0000000001 


0000000100 


0000000011 


0000000010 



Table 9-7 - coeff_token: totaLcoeff( ) / trailing_ones( ): Num-VLC_Chroma_DC 



trailing_ones( ) 


0 


1 


2 


3 


total_coeff( ) 










0 


01 








1 


000111 


1 






2 


000100 


000110 


001 




3 


000011 


0000011 


0000010 


000101 



9.1.6.2 Table selection 

For all elements, except chroma DC, a choice between three tables and one FLC is made. N is a value used for Table 
selection. Selection is done as follows: N is calculated based on the number of coefficients in the block above and to the 
left of the current block: N 0 and N L . In the table below, X means that the block is available in the same slice. The block's 
coding mode is not taken mto account when determining availability. When finding the block above and to the left for a 
block of Intra 16x1 6 DC coefficients, the location of the block is assumed to be (0,0), i.e. the upper left corner of the 
macroblock. 
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Tabl 9-8 - Calculation of N for Num-VLCN 



Upper block (Nu) 


Left block (NJ 


N 


X 


X 


(N L+ NuV2 


X 




Nu 




X 


N L 






0 



0 <= N < 2 : Num-VLCO 
2<=N<4 :Num-VLCl 
4 <= N < 8 : Num-VLC2 
N >= 8 : 6 bit FLC xxxxyy, as follows: 

As a part of the coeff_token, TotalCoeff-1 is transmitted in the first 4 bits (xxxx) and Tls is transmitted as the last 2 bits 
(yy). There is one exception: the codeword 00001 1 represents TotalCoeff=0. 

For chroma DC, Num-VLC_Chroma_DC is used. 

9.1.63 Decoding of level information: coeffjevel 

First, the sign of Tls are decoded from 1 bit each of trailing_ones_sign. A maximum of 3 bits are read. 

For the remaining level information, seven structured VLCs are used to decode levels. The structured level tables are 
explained in Tables 9-9 to 9-15. Lev-VLCO has its own structure while the other tables, Lev-VLCJV, N = 1 to 6, share a 
common structure. 



Table 9-9 - Level tables 



Lev-VLCO 


Code no 


Code 


LevelCode(±l,±2..) 


0 


1 


1 


1 


01 


-1 


2 


001 


2 


3 


0001 


-2 








13 


00000000000001 


-7 


14-29 


000000000000003 xxxx 


±8 to ±15 


30-> 


000000000000000 1 xxxxxxxxxxxx 


±16 -> 



For Lev-VLOV, N = 1 to 6, the following structure is used: 

let leveLcode be the level information to be decoded from the VLC tables, 

if (|level_code|-l) < (15«(AM)), 



Code: 0.. 
where 



.01x..xs, 

number of 0's = 

number of x*s = 

value of x's = 



(|leveLcode|-l)» (/V-l), 
AM, 

(|level_code|-l)%2 (yvi) , 

sign bit (0 - positive, 1 - negative) 



else, 
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28-bit escape code: 0000 0000 0000 0001 xxxx xxxx xxxs, 

where value of x's = (|leveLcode|-l) - (15«( N -I)), 

s = sign bit (0 - positive, 1 - negative) 



Table 9-10 - Level VLC1 



Lev-VL( 


:i 


Code no 


Code 


LevelCode (±1,±2..) 


0-1 


Is 


±1 


2-3 


01s 


±2 








28-43 


000000000000001s 


±15 


44 -> 


0000000000000001 xxxxxxxxxxxs 


±16 -> ~ J 



Table 9-11 - Level VLC2 



Lev-VLC 


2 


Code no 


Code 


LevelCode (±1, ±2..) 


0-3 


lxs 


±1 to ±2 


4-7 


Olxs 


±3 to ±4 




..s 




56-71 


00000000000000 lxs 


±29 to ±30 


72 -> 


0000000000000001 xxxxxxxxxxxs 


±31 -> 



Table 9-12 - Level VLC3 



Lev-VLC3 




Code no 


Code 


LevelCode (±I,±2..) 


0-7 


lxxs 


±1 to ±4 


8-16 


Olxxs 


±5 to ±8 








112-127 


000000000000001XXS 


±57 to ±60 


128 -> 


000000000000000 1 xxxxxxxxxxxs 


±61 -> 
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Table 9-13 - Level VLC4 



Lev-VLC4 


Code no 


Code 


LevelCode (±1,±2..) 


0-7 


lxxxs 


±1 to ±8 


8-16 


OIxxxs 


±9 to ±16 








112-127 


000000000000001 xxxs 


±113 to ±120 


128 -> 


000000000000000 1 xxxxxxxxxxxs 


±121 -> 



Table 9-14 - Level VLC5 



Lev-VLC5 


Code no 


Code 


LevelCode (±1, ±2..) 


0-7 


lxxxxs 


±1 to ±16 


8-16 


Olxxxxs 


±17 to ±32 








112-127 


00000000000000 1 xxxxs 


±225 to ±240 


128 -> 


0000000000000001 xxxxxxxxxxxxxs 


±241 -> 



Table 9-15 - Level VLC6 



Lev-VLC6 


Code no 


Code no 


LevelCode (±1,±2..) 


0-15 


1 xxxxxs 


±1 to ±32 


8-16 


01 xxxxxs 


±33 to ±64 








112-127 


00000000000000 1 xxxxxs 


±449 to ±480 


128-> 


0000000000000001 xxxxxxxxxxs 


±481 -> 



Normally all coefficient levels (coeffjevel) are equal to the decoded LevelCode value given in Tables 9-9 to 9-15. 
However, when Tls is less than 3, the level of the first coefficient (after Tls) is equal to the decoded LevelCode plus 1: 

If (first coefficient && trailing_ones() < 3) 

coeffjevel = (|level__code| + 1) * sign(level_code) 

else 

coeffjevel = leveLcode 

The last two entries in Lev-VLCO table are escape codes. The first escape code with 19 bits, four "x"*s, is used to decode 
the 8 levels above the last regularly coded level. The next escape code with 28-bits, 12 *V"s, is used to decode all 
remaining levels. For Lev-VLCl to Lev-VLC6 tables, only the 28-bit escape code is used. 
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9.1.63 Table selection 

Selections of the tables are changed during the decoding process based on number of coefficients, number of trailing 
ones, and the size of the previously decoded level value (coeffjevel). * 

Let VLC denote the Lev-VLCtf (N=0-6) to be used. After each level is decoded, the VLCN is updated according to 
the following method, where Level is the absolute value of the previously decoded level (coeffjevel). 
// VLC initialization for decoding first level ; 

if (total_coeff (coeff _token) > 10 && trailing_ones (coef f_token) < 3) 

Y LC = 1 // use Lev-VLCl for first level 

else 

V1 ' c =0 // use Lev-VXCO for first level 

// Assign FirstCoeff 

// Decode level_code here and assign coeff_level and Level 
// VLC update after decoding each level 

vlc_inc table[7] = {O, 3, 6, 12, 24, 48, 2 15 } 
if (Level > vlc_inc [VLC] ) 
VLC ++ 

if (FirstCoeff and Level > 3) 
VLC = 2 

^'^'SXAtZT^ wi,h u " v "° or ^ V - VLC1 whik ,te - - - — »• 

The same procedure is used for chroma AC and DC coefficient levels. 
9.1.6.4 Decoding of run information 

Run decoding is separated in total number of Zeros (i.e. the number of zeros located before the last non-zero coefficient 
in the zig-zag scan) and Run (of zeros) before each coefficient. 

9.1.6.4.1 Entropy Decoding of the total number of zeros: total_zeros 

The variable TotalZeros as given by total_zeros, is the sum of all zeros located before the last non-zero coefficient in a 
zig-zag scan. For example, given the string of coefficients 003004000020100 0, TotalZeros will be 2+244+1=9 
Since TotalCoeff is already known, it determines the maximum possible value of TotalZeros. One out of 15 VLC tables 
is chosen based on TotalCoeff. 

If TotalCoeff indicates that all coefficients are non-zero, TotalZeros is not decoded since it is known to be zero 



Table 9-16 - total_zeros tables for all 4x4 blocks 



TotalCoeff 
TotalZeros 


1 


2 


3 


4 


5 


6 


7 




0 


1 


111 


0101 


00011 


0101 


000001 


000001 




1 


011 


110 


111 


111 


0100 


00001 


00001 




2 


010 


101 


110 


0101 


0011 


111 


101 




3 


0011 


100 


101 


0100 


111 


110 


100 




4 


0010 


011 


0100 


110 


110 


101 


011 




5 


00011 


0101 


0011 


101 


101 


100 


11 




6 


00010 


0100 


100 


100 


100 


Oil 


010 




7 


000011 


0011 


011 


0011 


011 


oio 1 


0001 
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8 


000010 


0010 


0010 


011 


0010 


0001 


001 




9 


0000011 


00011 


00011 


0010 


00001 


001 


000000 




10 


0000010 


00010 


00010 


00010 


0001 


000000 


1 




11 


00000011 


000011 


000001 


00001 


00000 


- 


- 




12 


00000010 


000010 


00001 


00000 


- 


- > 


- 




13 


000000011 


000001 


000000 


- 


- 


- 


- 




14 


000000010 


000000 


- 


- 


- 










nnnoooooi 
















TO UalUOeL X. 

Total Zeros 


9 


9 


10 


11 


12 


13 


14 


15 


0 


000001 


000001 


00001 


0000 


0000 


000 


00 


0 


1 


0001 


000000 


00000 


0001 


0001 


001 


01 


1 




00001 


0001 


001 


001 


01 


1 


1 


- 


3 


011 


11 


11 


010 


1 


01 


- 




4 


11 


10 


10 


1 


001 


- 


- 


- 


5 t 


10 


o v oi 


01 


011 


- 


- 


- 


- 


€ 


010 


01 


0001 


- 


- 


- 


- 


- 


7 


001 


00001 


- 


- 


- 


- 


- 


- 


8 


000000 


- 


- 


- 


- 


- 


- 


- 


9 


- 


- 


- 


- 


- 


- 


- 


- 


10 


- 


- 


- 


- 


- 


- 


- 


- 


11 


- 


- 


- 


- 


- 


- 


- 


- 


12 


















13 



















Table 9-17 - TotalZeros table for chroma DC 2x2 blocks 



NumCoeff 
Total Zeros 


1 


2 


3 


(T 


1 


1 


1 


1 


01 


01 


0 


2 


001 


00 




3 


000 







9.1.6.4.2 Run before each coefficient 

At this stage it is known how many zeros are left to distribute (call this ZerosLeft). When decoding a mn.before for the 
first time, ZerosLeft begins at TotalZeros, and decreases as more run_before elements are decoded. 

For example, if there is only 1 zero left, the run before the next coefficient must be either of length 0 or 1, and only one 
bit is needed. 

The number of preceding zeros before each non-zero coefficient (called RunBefore) needs to be decoded to properly 
locate that coefficient Before decoding the next RunBefore, ZerosLeft is updated and used to select one out of 7 tables. 
RunBefore does not need to be decoded in the following two situations: 
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If the total number of zeros has been reached (ZerosLeft = 0) 

For the last coefficient in the reverse zig-zag scan. Then the value is known to be ZerosLeft. This also mean. th»t 
the maximum value to be decoded is 14. 



Table 9-18 - Tables for run_before 



Tot a 1 2jero«5 
Run Before 


1 


2 


J 


A 


J 


o 


>6 


0 


1 


1 


11 


11 


11 


11 


111 


1 


0 


01 


10 


10 


10 


000 


110 


2 


- 


00 


01 


01 


011 


001 


101 


3 


- 


- 


00 


001 


010 


011 


100 


4 


- 


- 


- 


000 


001 


010 


on 


5 


- 


- 


- 


- 


000 


101 


010 


6 












100 


001 


7 














0001 


8 














00001 


9 














000001 


10 














0000001 


11 














00000001 


12 














000000001 


13 














0000000001 


14 














00000000001 



9.2 
9.2.1 



Context-based adaptive binary arithmetic coding (CABAC) 

Decoding flow and binarization 

A binarization scheme provides a mapping of a non-binary valued alphabet of symbols onto a set of sequences of binary 
decisions, so-called bins. In subclauses 9.2.1.1 - 9.2.1.4 the elementary types of binarization schemes for CABAC are 
speciiied. 

£bdau^ a 9 2 n i ?-92?9 0din8 ^ ^ ^ ^ &nm&XA ° f binarization schemes for all syntax elements is given in 
9.2.1.1 Unary binarization 

l^J'J? f^T.^K^^'r c ^ wo [ ds , of the una *y code used for binarization of code symbols. For a code symbol C 
it consist of |C| 1 bits followed by the last bit with value '0\ The first bin number corresponds to the first bit of the 
unary codeword with increasing bin numbers towards the last bit, as shown in Table 9-19. 



Table 9-19 - Binarization by means of the unary code tree 



Code 
symbol 


Binarization 


0 


0 














1 


1 


0 












2 


1 


1 


0 











.1100641A_»_> 



ITU-T Rec. H.264 (2002 E) 



93 



ISO/IEC FCD 14496-10 



3 


• 


1 


1 


0 








4 


1 


1 


I 


1 


0 






5 


1 


1 


1 


1 


1 


0 




bin_num 


1 


2 


3 


4 


5 


6 





9.2.1.2 Truncated unary (TU) binarization 

The truncated unary (TU) binarization is defined for a finite alphabet {ft...,^} by applying the unary binarization of 
subclause 9.2.1.1 to all code symbols C with C < C^; the binarization of the symbol C^C^ is defined by a code word 
consisting of ' 1 *s (without the last bit of value *0*). Numbering of the bins is the same as for unary binarization. 

9.2.13 Concatenated unary/ k^-order Exp-Golomb (UEGk) binarization 

Concatenated unary/k lh -order Exp-Golomb (UEGk) binarization consists of a concatenation of a prefix code word and a 
suffix code word. The prefix is formed by using a truncated unary binarization with C^^ UCoff where UCoff denotes 
the cut-off parameter. For all code symbols C with C < UCoff, the suffix code word is void; for code symbols C with C > 
UCoff the suffix consists of an Exp-Golomb code of order k for the symbol C-UCoff For a given symbol S the Exp- 
Golomb code of order k is constructed as follows: 

while (1) { 

//first unary part of EGk 

if (symbol >= (unsigned int) (l<<k) ) { 

put ( » 1 ' ) ; 

S = S - (l<<k) ; 

} 

else 

put('O'); //now terminating zero of unary part of EGk 

while (k--) //finally binary part of EGk 

put ( (S»k) & 0x01 ) ; 
break; 

} 

} 

The numbering of bins is such that bin number bin_num = 1 relates to the first binary decision in the unary prefix code 
with increasing numbers towards the least significant bits of the binary part of the Exp-Golomb suffix. 

9.2.1.4 Fixed-length (FL) binarization 

Fixed-length (FL) binarization is applied to a finite alphabet {(),..., C^} of code symbols. It is constructed by using a 
L-bit binary representation of a code symbol, where L= loglC^ J+l. The numbering of bins for the fixed-length 
binarization is such that the bin_num = 1 relates to the least significant bit with increasing bin numbers towards the most 
significant bit. 

9.2.1J5 Binarization schemes for macroblock type and sub macroblock type 

The binarization scheme for decoding of macroblock type in I slices is specified in Table 9-20. If 
adaptive J>lock_size_transform_flag = = 1, the binarization for decoding of macroblock type in I slices is specified in 
Table 12-10. 

For macroblock types in SI slices the binarization consists of a prefix and a suffix part The prefix consists of a single bit 
b ]= ((mbjype = = Slntra_4x4) ? 0 : 1). The suffix part for mb_type Sintra_4x4 is void, while the suffix parts for all 
other macroblock types are given by the corresponding binarization pattern specified in Table 9-20. 

The binarization schemes for P, SP, and B slices are specified in Table 9-21. The binarization for intra macroblock types 
in P and SP slices corresponding to mb_type values 7 to 30 consists of a prefix as specified in Table 9-21 and a suffix as 
specified in Table 9-20 for the corresponding intra mb_type. For intra macroblock types in B slices (mb_jype values 23 
to 47) the binarization consists of a prefix as specified in Table 9-21 and a suffix as specified in Table 9-20 for the 
corresponding intra mb.type. If adaptive_block_size_transform_flag = = 1, the same prefix is used as specified in Table 
9-21 for intra macroblock types of the corresponding slice type. However, the corresponding suffix parts are specified in 
Table 12-10. 

For P, SP, and B slices the specification of the binarization for sub_mb__type is given in Table 9-22. - 
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Table 9-20 - Binarization f r macroblock types for I slices 



Value (name) of mb_type 


Binarization 


0 (Intra_4x4) 


0 












1 (IntnU6xl6__0_(L0) 




o 


0 


o 


o 




2 (Intra_16xl6_l_0_0) 




o 


o 


o 


] 




3 (IntnU6xl6_2_0_0) 




0 


o 


I 


o 




4(Intra_16xl6_3JL0) 




0 


0 


1 


1 




5(Intra_16xl6_0_l_0) 




o 




o 


0 


A 
\J 


6(Intra__16xl6_l_l_0) 




0 




o 


o 


1 
1 


7(Intra_16xl6_2_l_0) 




o 




o 


] 


A 

V 


8(IntnU6xl6_3_l_0) 




o 




o 


1 

1 


1 
I 


9 (lntra_ 1 6x 1 6_0_2_0) 




o 




1 

1 


A 

u 


A 


1 0 (Intra_l 6x 1 6_ 1 _2_0) 




o 




1 
I 


A 

u 


1 
I 


11 (Intra_16xl6_2_2J)) 




ft 




1 
I 


1 

1 


U 


12 (Intra_16xl6_3_2_0) 




A 
\J 




1 
I 


1 


1 


13 (Intra_16xl6_0_0_l) 






A 
\J 


A 

u 


u 




14 (Intra_I6xl6_l_0_l) 






A 

u 


u 


1 




15 (Intra. 16xl6_2_0_l) 






A 

'J 


i 


U 




16 (Intra_16xl6_3_0__l) 








i 
i 


1 
1 




17(Intra_16xl6_0_l_l) 








A 
\J 


A 
KJ 


A 
U 


18(Intra_16xl6_l_l_l) 








o 


A 

V 


1 
1 


19(Intra_16xl6_2_J_l) 








o 


J 


A 
\J 


20 (Intra_16xl6_3_l_l) j 1 






o 




I 


21 (IntraJ6xl6J)_2_l) 








1 


0 


0 


22 (Intra_16xl6_l_2„l) 








1 


0 




23 (IntnU6xl6_2_2_l) 








1 


1 


0 


24(Intra_16xl6_3_2_l) 








1 


1 


1 


bin_num 




2 


3 


4 


5 


6 



Table 9-21 - Binarization for macroblock types for P, SP, and B slices 



Slice type 


Value (name) of mb_type 


Binarization 




0(Pred_L0_16xl6) 


0 


0 


0 












1 (Pred_L0_L0_16x8) 


0 


1 


1 










P, SP slices 


2(Pred_tt)JLXL8xl6) 


0 


1 


0 










4 (Pred_8x8) 


0 


0 


1 












6 (Pre<L8x8refO) 


na 




7 to 30 (Intra, prefix only) 


1 










1 
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0(DirecCl6xl6) 


0 
















1 (Pred_L0_16xl6) 


■ 


0 


0 












2 (BiPred_Li_16xl6) 


■ 


0 


1 












3 (BiPred.BL16xl6) 




1 


0 


0 


0 


0 






4(Pred_LO_LO_16x8) 




■ 


0 


0 


6 


1 






5 (Pred_L0__L0_8xl6) 






0 


0 


1 


0 






6 (BiPred_Ll_Ll_16x8) 




1 


0 


0 


1 


1 






7 (BiPred_Ll_Ll_8xl6) 


» 




0 


1 


0 


0 






8 (BiPred_L0_Ll„16x8) 






0 


1 


0 


1 






9(BiPredJU0_Ll_8xl6) 


» 




0 


1 


1 


0 






10(BiPred_Ll_L0_16x8) 






0 


1 


1 


1 




tj slices 


11 (BiPred_Ll_LO_8xl6) 






■ 




1 


0 




12 (BiPred_L0_BL16x8) 








0 


0 


0 


0 




13 (BiPred_L0_Bi_8xl6) 






I 


0 


0 


0 


1 




14 (BiPred_Ll_Bi_16x8) 








0 


0 


1 


0 




15(BiPred_L13i_8xl6) 








0 


0 


1 


1 




16 (BiPred_Bi_L0_16x8) 




> 




0 


1 


0 


0 




1 7 (BiPred_Bi_L0_8x 1 6) 








0 


1 


0 


1 1 




18 (BiPred_Bi_Ll_16x8) 






1 


0 


1 


1 


0 




19 (BiPred_BLLl_8xl6) 






1 


0 


1 


1 


1 




20 (BiPre<LBi_Bi_16x8) 








1 


0 


0 


0 




21 (BiPred_Bi_BL8xl6) 






1 


1 


0 


0 


1 




22 (BiPred_8x8) 


1 


1 


1 


1 


1 


1 






23 to 47 (Intra, prefix only) 








1 


0 


1 




bin_num 






2 


3 


4 


5 


6 


7 



Table 9-22 - Binarization for sub macroblock types in P and B slices 



Slice type 


Value (name) of 
sub_mb_type 


Binarization 




0 (Pred JL0_8x8) 
















1 (Pred_L0_8x4) j 


0 


0 


0 








P slices 


2 (Pred_L0_4x8) 


0 


0 


1 


1 








3 (Pred_L0_4x4) 


0 


0 


1 


0 








4 (Intra_8x8) 


0 


1 










B slices 


0 (Direct_8x8) 


0 














I (Pred_L0_8x8) 


1 


0 


0 










2 (BiPred_Ll_8x8) 


1 


0 


1 
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J (DiFred_Bi_5xo) 






0 


0 


0 






4 (rrea_LAJ_ox4) 






0 


0 


1 






5 (Pred_L0_4x8) 






0 


1 


0 






o (,t>irrea_Li_ox4) 






0 


1 


1 






/ ^Dirrea_Li_4xoj 






1 


0 


0 


0 




o (t5irrea_Di_oX4 < ; 






1 


0 


0 


1 




y voirreo__xJi_4xo < ) 






] 


0 


1 


o ! 




in fPr**H 1 n /iv/i\ 

1U v,rTcu_LU_4X4J j 






1 


0 


1 


l 










1 


1 


0 


0 




12(BiPred_Bi_4x4) 






1 


1 




i 
i 




13 (Intra_8x8) j 






1 


1 






bin_num J 




2 


3 


4 


5 


6 



9.2.1,6 Decoding flow and assignment of binarization schemes 

In this subclause, the binarization schemes used for decoding of coded_block_pattern, delta^qp, the syntax elements of 
reference picture index, motion vector d!ata, Intra4x4 prediction modes and are specified. 

The coded block pattern information is decoded using the relationship as given in subclause 7.4.6: coded_block_pattern 
= coded_block_patternY + 16*nc. In a first step, the Iuma part coded_block_patternY of coded_block_pattern is decoded 
using the fixed-length (FL) binarization with = 15 and L = 4. Then, the chroma part nc is decoded using TU 
binarization with C max = 2. 

Decoding of the delta_qp parameter is a two-step process. First, an unsigned code value wrapped_delta_qp > 0 is 
decoded using the unary binarization. Then, wrapped_delta_qp is mapped to the signed value of the delta_qp parameter 
according to the relationship given in Table 9-2. 

The decoding process for spatial intra prediction modes associated with luma of macroblock type Intra_4x4 and 
Sintra_4x4 is as follows. First, a parameter intra_pred_indicator is decoded using the truncated unary (TU) binarization 
with C max = 8. If intra_pred_indicator = 0, use_most_probable_mode is set to 1. If intra_pred_indicator > 1, 
remaining_mode_selector = intra_pred_indicator - 1. Given the parameters most_probable_mode and 
remaining_mode_selector, the intra prediction mode intra_pred_mode is obtained in the same way as specified in 
subclause 9.1.5.1. The decoding order of the prediction modes is the same as shown in Figure 9-1 b). For decoding of 
the spatial intra prediction mode for chroma intra__chroma_pred_mode, the truncated unary (TU) binarization with 
Cn« = 3 is used. 

The reference picture index parameter is decoded using the unary binarization as given in subclause 9.2.1.1. 

Each component of the motion vector data-is decoded separately starting with the horizontal (h) component. First the 
absolute value abs_mvd_comp and then the sign sign_mvd_comp of each component (comj?=h r v) shall be decoded. The 
binarization scheme applied to abs_mvd_comp is given by the concatenated unary/3 r -order Exp-Golomb (UEG3) 
binarization with cut-off parameter Ucoff = 9. Note that for decoding the Exp-Golomb suffix the decoder bypass 
Decode_eq_prob as specified in subclause 9.2.4.3.5 is used. 

9.2.1.7 Decoding flow and binarization of transform coefficients 

Decoding of transform coefficients is a three-step process. First, the one-bit coded_block_flag is decoded for each block 
of transform coefficients unless the codedJ>lock_pattern symbol on macroblock level indicates that the regarded block 
has no non-zero coefficients. If the coded__block_flag symbol is zero, no further information has to be decoded for the 
block. Otherwise, it is indicated that there are significant coefficients inside the block. The latter case implies that, in a 
second decoding step, for each scanning position i except the last position in a block the binary-valued 
significant_coeff_flag[i] has to be decoded. If significant_coeff_flag[i] has the value of one, the corresponding position i 
in the block contains a significant coefficient and a further binary-valued last_significant_coeff_fIag[i] is decoded. If 
last_significant_coeff_flag[i] is zero, there is at least one further significant coefficient to be decoded; otherwise, the last 
significant coefficient along the scanning path is reached. If this is the case, the absolute value minus 1 
coeff_absolute_value_minus_l and then the sign of the coefficient coeff_sign is decoded for each significant transform 
coefficient by traversing the block in reverse scanning order. coeff_absolute_value_minus_l is decoded using the 
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concatenated unary/zero-order Exp-Golomb (UEGO) binarization with UCoff=14. Similar to the decoding of absolute 
values of the motion vector components, the Exp-Golomb suffix is decoded by using the decoder bypass 
Decode_eq_prob . 

9.2.1.8 Decoding of sign information related to motion vector data and transform coefficients 

Decoding of th sign information signjmvdjcomp of the motion vector components and coeffjsign of the levels 
corresponding to significant transform coefficients is performed as follows. Using the decoder bypass Decode_eq_prob 
as specified in subclause 9.2.4.3.5 first a binary indicator signjnd is decoded. Then the sign information signjnfo is 
recovered by signjnfo - ((signjnd - = 0) ? 1 : -J). 

9.2.1.9 Decoding of macroblock skip flag and end-of -slice flag 

Decoding of the mb_skip_flag is as follows: First, a binary-valued mb_skip^flag_decoded is decoded using the context 
model as specified in subclause 9.2.2.2. In a second step, the actual value of mb_skip_flag is obtained by inverting 
rnb_skip_flag_decoded, Le„ mb_skip_flag = mb_skip_flag_decoded A 0x01. 

The end_of_slice_flag is decoded using a fixed, non-adaptive model by chosing State = 63 and MPS = 0. The following 
mechanism guarantees a fixed model, although the coding engine uses a probability estimator after each decoding step as 
further specified in subclause 9.2.4.2. By observing a sequence of end__of__slice values 4 0* meaning that the end of a slice 
has not been reached, the initial chosen state will not be altered, since for the observation of a MPS symbol the state 
variable State = 63 will be mapped onto itself in the probability estimation. However, as soon as a LPS value of 4 1* is 
decoded for end_of_slice_flag, the probability estimation for the LPS will not affect the subsequent decoding process, 
because the end of a slice is reached, and all context models are refreshed using the initial states. 

9.2.2 Context definition and assignment 

For each bin number, a context variable is defined by a conditioning term containing prior decoded symbols or parts 
thereof. The possible numerical values of a context variable specify the different context models associated with a 
specific bin number. Typically, there are several possible values or context labels for each bin number bin_num. 
However, in some cases the context variable may simply be a constant label, in which case there is only one fixed 
context model. 

This subclause defines first a variety of generic types of context variables, so-called context templates, for conditional 
coding of syntax elements. Then, for each bin number of a syntax element, the specification of the corresponding context 
variable is given. For the different bin numbers associated with the binarization of a given syntax element or parts 
thereof, a unique context identifier context_id is chosen such that the context variable associated to bin number k is given 
by context_id[k]. Since there is always a maximum bin number N with a context variable context_id[N] that is different 
from the corresponding context variable context_id[N-l] for the preceding bin number N-l, it is sufficient to specify a 
context identifier for each index k with l£ k< N, where N is called the maximum index of the context identifier 
maxjidx_ctx_id. 

Table 9-23 provides an overview of the context identifiers associated to each category of syntax elements. A detailed 
description of the corresponding context variables is given in the subsequent subclauses. Note that each context identifier 
corresponds to a unique range of context labels, which, in the case of macroblock type, may overlap for the different 
slice types I, SI, P, SP, and B. 

If adaptive_block_size_transform_flag = = 1, the context identifiers related to decoding of transform coefficients utilize 
an additional set of ranges as further specified in Table 12-12. 



Table 9-23 - Syntax elements and associated context identifiers 



Syntax element 


Context identifier 


Type of 
Binarization 


max_idx_ctx_id 


Range of 

context 

label 


mb_skip_flag 


ctx_mb_skjp 


-/- 


1 


0-2 


mbjype 


ctx__mb_type_I 


Table 
9-14 


5 


0-7 




ctx_mb_type_S I_pref 


-/- 


1 


0-2 




ctx_mb_type_SI_su f 


Table 
9-14 


5 


3-10 




ctx_mb_type_P 


Table 


3 


3-6 
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ctx_mb_type_B 


9-15 


4 


i ft 
J — o 


ctx_mb_ty pe_P_su f 


Table 


5 


O — y I 


ctx_mb_type_B_suf 


9-14 




8-11 


ctx_b8_mode_P 


Table 


4 


12-15 1 


ctx_b8__mode_B 


9-16 


4 


12-15 j 


mvd JO and mvdj 1 


ctx_abs_mvd__h 


UCoff=9 


5 


16-22 


ctx_abs_mvd_v 


UEG3, 
UCoff=9 


5 


23 - 29 | 


refjdxJO and ref_idx_ll 


ctx_ref_idx 


Unary 


3 


30-35 [ 


delta_qp 


ctx_delta_qp 


Unary 


3 


36-39 J 


chroma_prcd_modc 


ctx_ipred_chroma 


Til 


3 


40-44 


intra_pred_mode 


ctx_ipred_luma 


TU, 


2 


45-62 


coded_block pattern 


ctx_cbp_luma 


Pi 

^,=15 


4 


63-66 




ctx_cbp_chroma 


TU, 
C — ? 


2 


67-74 ! 


coded_block_flag 


ctx_cbp4 


_/_ 


-/- 


75-94 


signifjcant_coeff 


ctx_sig 


-/- 


-/- 


95-155 J 


last_coeff 


ctx_Jast 


-/- 


-A 


156-216 j 


coeff_absolute_value_minus_l 


ctx ahs IpvpI 


UEGO, 
UCoff=14 


2 


217-266 


end_of_slice_fl ag 


ctx_eos 


Fixed, non- 
State(ctx_eos) = < 


adaptive model 
53, MPS(ctx_eos) = ( 


with 1 
) j 



9.2.2.1 Overview of assignment of context labels 

Tables 9-24 and 9-25 contain all context identifieis along with their corresponding range of context labek Th* 
assoctauon of context labels (modulo some offset) and bin numbers shows which contexf variable uses ^ fixed moL ™h 
cttexMa e heS eS 3 T " ^ ^ " «* < hoSe en^es wheS ate of^em 

fTaSe 9 Sfr^ ST? 7 * -V *"* bi "- nUm <™° 9 ' 24) ° r block de P endent c °<"<*t ca tego"y 

( 1 able 9-25). These context variables are specified in the following subclauses. -«-«»egory 

Table 9-24 - Overview of context identifiers and associated context labels 



Context identifier 


Range of 

context 

label 


Offset for 
context label 


max_idx_ctxjd 


bin_num 


1 


2 


3 


4 


> 5 


ctx__mb_skip 


0-2 


0 


1 


0,1,2 


-/- 


-/- 


-/- 


-/- 


ctx__mb_type_I 


0-7 


0 


5 


0,1,2 


3 


4 


5,6 


6,7 


ctx_mb_type_SLpref 


0-2 


0 


1 


0,1,2 


-/- 


-A 


-/- 


-/- 


ctx_mb__type_SI_suf 


3-10 


3 


5 


0,1,2 


3 


4 


5,6 


6,7 


ctx__mb_type_P 


3-6 


3 


3 


0 


1 


2,3 * 


-/- 


-/- 


ctx_mb_type_P_su f 


6-9 


6 


5 


0 


1 


2 


2,3 


3 j 
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ctx_mb_type_B 


3-8 


3 


4 


0,1,2 


3 


4,5 


5 


5 


clx_mb_type__B_suf 


8-11 


8 


5 


0 


1 


2 


2,3 


3 


ctx_b8_mode_P 


12- 15 


12 


4 


0 


1 


2 


3 


-/- 


ctx_b8„mode_B 


12- 15 


12 


4 


0 


1 


2,3 


3 


3 


ctx_abs_mvd_h 


16-22 


16 


5 


0,1,2 


3 : 


4 


5 


6 


cu_abs_mvd_v 


23-29 


23 


5 


0,1,2 


3 


4 


5 


6 


ctx rcf idx 


30-35 


30 


3 


0,1.2.3 


4 


5 


5 


5 


ctx_delta_qp 


36-39 


36 


3 


0,1 


2 


3 


3 


3 


cu_ipred_chroma 


40-44 


36 


3 


0,1.2 


3 


4 




-/- 


ctx_ipred_luma 


45-62 


42 


2 


0,.„,8 


9.....17 


9,...,17 


9.....17 


9,...,!7 


ctx_cbp_luma 


63-66 


60 


4 


0,1,2,3 


0,1,2,3 


0,1,2,3 


0,1,23 


-/- 


ctx_cbp_chroma 


67-74 


64 


2 


0,1,2,3 


4,5,6,7 




-/- 


-/- 


ctx_absjevel 


217-266 


217+ 

1 O*context_category 


2 


0,...,4 


5,...,9 


5 9 


5,...,9 


5 9 



Table 9-25 - Overview of context identifiers and associated context labels (continued) 



Context 
identifier 


OITset (range) of 
context label 


context_category (as specified in Table 9-22) 


0 


1 


2 


3 


4 


ctx_cbp4 


75(75 - 94) 


0-3 


4-7 


8-11 


12-15 


16- 19 


ctx_sig 


95 (95-155) 


0- 14 


15-28 


29-43 


44-46 


47-60 


ctxjast 


156(156-216) 


0-14 


15-28 


29-43 


44-46 


47-60 



Figure 9-2 - Illustration of the generic context template using two neighbouring symbols A and B for conditional 

coding of a current symbol C 



9.2.2.2 Context templates using two neighbouring symbols 

The generic design of this type of context variable is shown in Figure 9-2. It involves two previously decoded symbols or 
bins of the same syntax element that correspond to the spatially neighbouring blocks to the left (A) and on the top (B) of 
the regarded block C. The generic form of the equation defining this type of context is given by 

ctx_yar_spat - cond_term(A, B) 9 (9-1) 



where the conditioning term cond_term(A, B) describes the functional relationship between the spatially neighbouring 
symbols A and B, on the one hand, and the values of the context variable, on the other hand. Three special cases of this 
template are specified as follows: 

ctx_varjspatl - condjterm(A) + cond_term( B), (9-2) 
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ctx_var_spat2 = cond_term(A) + 2*cond_term( B), 
ctx_var_spat3 = cond_term(A). 



(9-3) 
(9-4) 



Table 9-26 contains the specification of context variables relying on templates using two neighbouring symbols The 
conditioning term of the context variable ctx_cb P 4 depends on six block types as given in Table 9-28 (Luma-DC, Luma- 
AC, Chroma-U-DC, Chroma- V-DC, Chroma-U- AC, Chroma-V-AC). 

Table 9-26 - Specification of context variables using context templates according to Equations (9-2) - (9-4) 



Context variable 


Context template 


cond_term(X), 
semantics of X 


cond_term(X), 
if X not available 


ctx_mb_skip 


ctx_yarjspatl 


(mb_skip_flag(X) = = 0) ? 1: 0 
X: neighbouring macroblock 


0 


cU_mb_type_I[l] 


ctx_var_spatl 


(mb_type(X) != Intra_4x4) ? 1: 0 
X: neighbouring macroblock 


0 


ctx_mb_type_SLpref[ 1 ] 


ctx_var_spatl 


(mb_type(X) != Sintra_4x4) ? 1: 0 
X: neighbouring macroblock 


0 


ctx_mb_type_S I_sufl 1 ] 


ctx_var_spatl 


(mb_type(X) != IntraL_4x4) ? 1: 0 
X: neighbouring macroblock 


0 


ctx_mb_typeJB[l] 


ctx_yar__spatl 


((mb_type(X) != Direct)? 1:0) 
X: neighbouring macroblock 


0 


ctx_ipred_chroma[ 1 J 


ctx_var_spatl 


((intra_chroma_pred_mode(X) != 0) ? 1: 0) 
X: neighbouring macroblock 


0 


ctx_ref_idx[l] 


ctx_var_spat2 


(ref.idxJ0/ref_idx_ll(X) != 0) ? 1: 0 
X: neighbouring block 


0 | 


ctx_ipred_Iuma[i], i=l,2 


ctxj»arjspat3 


9*(i-l) + intra_pred_mode(X) 
X: neighbouring block 


o ! 


ctx_cbpjuma[i], i=l 4 


ctx_var_spat2 


i-th bit of coded_block_patternY(X) 
X: neighbouring 8x8 block of i-th block 


0 j 


ctx_cbp_chroma[ 1 ] 


ctx_yar_spat2 


(nc(X) != 0)?1: 0 

X: neighbouring macroblock 


0 


ctx_cbp_chroma[2J 


ctx_yar_spat2 


4 + (nc(X) == 2)? 1:0 
X: neighbouring macroblock 


0 


ctx_cbp4 


ctx_yar_spat2 


coded__block__pattern_4(X) 

X: neighbouring block of same block type 


1 , if X is INTRA; 1 
0, otherwise 1 


ctx_delta_qp 


ctx_var_spat3 


(delta_qp(X) != 0) ? I: 0 
X: neighbouring macroblock 


0 [ 



The specification of the context variables ctx_abs_mvd__h[l] and ctx_abs_mvd_v[l] is given as follows. 

0, (\mvd _ comp(A)\ + \mvd _ comp(B)\) < 3, 



ctx „ abs _ mvd _ comp[l ) = 



2, (\mvd _ comp(A)\ + \mvd _ comp(B)\) > 32, 
^otherwise, 



(9-5) 



where comp . denotes the component h (horizontal) or v (vertical) and A, B denote the neighbouring blocks of the block to 
decode as shown in Figure 9-2 Since the neighbouring blocks A and B may belong to different macroblock partitions, 
the followmg principle for identifying the proper neighbouring blocks that are used in Equation (9-5) is established First 
J*LT°^ n VeCt ° r I* aSSUm6d l ° bC given oversam P ,ed form such that each 4x4 block has its own motion vector 
(MVy That means on the one hand, that in case of a neighbouring block having a coarser partition, the related 4x4 sub- 
blocks are assumed to inherit the MV from the corresponding parent block(s) in the quadtree partition. On the other hand 
if the current block C represents a larger block than a 4x4 block, it is assumed to be represented by the corresponding 
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leftmost 4x4 sub-block in the top row of 4x4 sub-blocks. Then, given a block C, the neighbouring 4x4 sub-blocks B on 
top and A to the left of the representing 4x4 sub-block of C are chosen for evaluation of Equation (9-5). 

9.2.23 Context templates using preceding bin values 

Let (bi, - .., b N ) denote the binarization of a given symbol C. Then, a generic type of context variable associated with the 
k-th bin of C is specified as 

ctx_yar_bin[k] = cond_term(b h .., t b k -il (9-6) 

where l<k<N. In Table 9-27, the specification of context variables using this type of context template is given. 

Table 9-27 - Definition of context variables using the context template according to Equation (9-6) 



Context variable 


cona_term(*j,. 


ctx_mb_type_I[4] 


(b 3 = = 0)?5:6 


ctx_mb_type_I[5] 


(b 3 = = 0)?6:7 


ctx_mb_type_SI_sufl4] 


(b 3 = = 0) ? 5: 6 


ctx_mb_type_S I_sufI5] 


(b3 = = 0)?6:7 


ctx_rab_type_P[3) 


(b2 = = 0)?2:3 


cU_mb_type_P_sufI4] 


(b 3 = = 0)?2:3 


ctxJmb_typc_B[3] 


(b2 = = 1)?4:5 


ctx_mb_type_B_sufI4] 


(b 3 = = 0) ? 2: 3 


ctx_b8_raode_B[3] 


(b2== 1)?2:3 



9.2.2.4 Additional context definitions for information related to transform coefficients 

Three different additional context identifiers are used for conditioning of information related to transform coefficients. 
All these three types depend on context categories of different block types denoted by the variable context _cate gory. The 
definition of these context categories is given in Table 9-28. 

Table 9-28 - Context categories for the different block types 



block_type 


MaxNumCoefT 


context_category 


Luma DC block for INTRA 16x1 6 mode 


16 


0: Luma- Intra 1 6-DC 


Luma AC block for INTRA 16x1 6 mode 


15 


t:Luma-Intral6-AC 


Luma block for INTRA 4x4 mode 


16 


2. Luma -4x4 


Luma block for INTER 4x4 mode 


16 


U-Chroma DC block for INTRA mode 


4 


3:Chroma-DC 


V -Chroma DC block for INTRA mode 


4 


U -Chroma DC block for INTER mode 


4 


V-Chroma DC block for INTER mode 


4 


U-Chroma AC block for INTRA mode 


15 


4:Chroma-AC 


V-Chroma AC block for INTRA mode 


15 


U-Chroma AC block for INTER mode 


15 


V-Chroma AC block for INTER mode 


15 
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Additional context categories are used in the case of adaptive_bIock_size_transform flag = = j as soecif.ed i„ 
subclause 12.5.2. The context identifiers ctx.sig and ctxjast are related to the binary vaTued information ofSIO fl „H 
r^dei block: 6 ' C ° nteXt " addki ° nal 0n ,he ^^saion iSSJ^ISS 



ctx_sig[scanning_pos] = Map_sig( scanning_pos), 
ctx_last[scanning_pos] = Map_last(scanning_pos). 



(9-7) 
(9-8) 



The specifiaction of Map_sig and Map_last in Equations (9-7) and (9-8) depends on the block type. For context category 
0 - 4 the corresponding maps are given by the identity, i.e., wmexi_caiegory 

Map_sig(scanning_pos) = MapJast(scanning_pos) = scanning_pos, if context_category = 0.....4, 

where scanning_pos denotes the position related to the zig-zag scan. For the additional context categories 5 - 7 which 
sTbdate" 2 T2 Pt,Ve - b,0Ck - S,2e ^ anSf0rm - fla8 == h * e of M a P _sig and M^J^ is 

£rSl3 ? abs - leVe, 7 m K 1, abSO ' Ute Va,UCS ° f si « nificant ^nsform coefficients minus 1, the context identifier 
as follows 1 mg ° C ° nleXt VariaWeS CtX - abs - ,eve, l" and coc_absJeve U 2] is used, which ie defined 



ctx_absjev[l] = ((num_decod_abs_lev_gtl>=0) ? 4: min(3, num_decod_ab S _lev_eql)), 
ctx_abs_lev[2] = min(4, num_decod_abs_lev _gtl), 



(9-9) 
(9-10) 



where num_decod_abs Jev_eql denotes the number of decoded coefficients with magnitude equal to 1 and 
nurn decod_abs_le^tl denotes the number of decoded coefficients with magnitude greater than 1 Both numbl'rs^ 
related to the same transform coefficient block, where the current decoding take? place, which meZ Ttat no 
addmonahnformation outs.de the regarded transform coefficient block is used for the context variables ctxTsllevelfk]! 

^•2.3 Initialisation of context models 

9.23.1 Initialisation procedure 

^ t p th nl be?inni rl° f e3Ch S " C !' !f h °° nteXt m ° dei is initialised wi «> an initial state, which consists of a state number and 
tte meamng of the most probable symbol (MPS) as further described in subclause 9.2.4.2. The actual in 1^1 sTate of a 
context model depends linearly on the (initial) quantization parameter QP of the given slice, such that fo ^ Sch context 
model a pa.r of table entries {m, n) is given, from which the initial state is computed in the foilowTng way 
1 - Compute pre_state = (( m*(QP-12))»4) + n; 

2 ' nrp m l^- S,ate l ° rangC °f,£ 10,) f ° r P - a " d B - S,iceS and to the ran S e of f 27 ' 74 ] I-slices, i.e 
pre_state = mm (101, max(0,pre_state)) for P- and B-slices and 
pre_state = min (74, max(27,pre_state)) for I-slices; " d 

3. Map pre.state to {state, MPS} pair according to the following rule- 
if (pre_state <= 50) then {state = 50-pre_state, MPS = 0} else {state = pre_state-51, MPS = 1 } 

9.2.3.2 Initialisation procedure 

Iff.?" 29 ?" 34 C ° nta u the initialisation Parameters related to the context variables of all syntax elements from 
which the initial states can be obtained by using the rules given in subclause 9.2.3. 1 . elements, trom 

Table 9-29 - Initialisation parameters for context identifiers ctx_mbjypej, ctx_mb_type_SI_pref, , 



Context 
label 


ctx_mb 




ctx_mb_type_SI_pref 


ctx_mb_skip 






m 


n 


m 


n 


m 


n 


m 


n i 


m 


n 
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0 


7 


25 


7 


25 


-23 


66 










, 


8 


35 


8 


35 


-14 


77 










2 


-2 


63 


-2 


63 


-9 


88 
















ctx_mb_type_SI_suf 






ctx_mb_ 


-type_P 


ctx_mb_ 


_typc_B 


3 


-9 


68 


7 


25 






2 


13 


9 


49 


4 


-15 


74 


8 


35 






14 


24 


3 


65 


5 


-3 


36 


-2 


63 






-21 


69 


0 


78 


6 


-1 


51 


-9 


68 






-1 


52 


-13 


81 


7 


0 


50 


-15 


74 










-14 


73 


8 






-3 


36 










-8 


64 


9 






-1 


51 














10 






0 


50 















Tab! 9-30 - Initialisation parameters for context identifiers ctx_b8_mode_P, ctx_b8_mode_B, ctx_mb_type_P_suf, 

ctx_mb_type_B_suf 



' Context 
label 


ctx_mb_type_P_suf 
ctx_mb_type_B_suf 


Context 
label 


ctx_b8_mode_P 


ctx_b8_ 


mode_B 


m 


n 


m 


n 


m 


n 


9 


-9 


55 


12 


8 


46 


-9 


62 


10 


-7 


50 


13 


12 


11 


-12 


66 


11 


2 


47 


14 


-4 


62 


-9 


56 








15 


18 


48 


3 


47 



















Tabic 9-31 - Initialisation parameters for context identifiers ctxjabs_mvdji, ctx_abs_mvd_v, ctxjrefj 



Context 


ctx_abs_mvd_h 


Context 


ctx_abs_mvd_v 


Context 


ctx_rc 


fjdx 


label 


m 


n 


label 


ra 


n 


label 


m 


n 


16 


1 


48 


23 


-1 


45 


30 


3 


27 


17 


-5 


60 


24 


-5 


59 


31 


-1 


47 


18 


-8 


70 


25 


-9 


71 


32 


0 


45 


19 


2 


52 


26 


0 


50 


33 


-2 


60 


20 


2 


| 62 


27 


3 


61 


34 


-1 


57 


21 


-3 


64 


28 


-3 


63 


35 


0 


48 


22 


-2 


80 


29 


1 


80 









Table 9-32 - Initialisation parameters for context identifiers ctx_delta_qp, ctx_ipred_chroma, ctxjpred Junta 



Context 
label 


ctx_delta_qp 


Context 


ctx_ipred_Jtima 


Context 


ctx_iprt 


rd_luma 


m 


n 


label 


m 


n 


label 


m 


n 


36 


0 


28 


45 


-5 


49 


54 


-4 " 


61 
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37 


o 




*KJ 




58 


55 


-6 


64 


38 


0 


50 


47 


-j 


58 


56 


-6 


63 


39 


0 


50 


48 


A 


58 


57 


-3 


75 




die iniw 


Luiuiiid 




-5 


59 


58 


-4 


63 


40 


-5 


50 


50 


A 


£.f\ 

60 


59 


-7 


65 


41 


0 


50 


5J 


-v> 


ol 


60 


-1 


63 


42 


0 


50 


52 


-5 


62 


61 


-18 


66 


43 


0 


50 


53 


-4 


60 


62 


-9 


67 


44 


0 


50 















Table 9-33 - Initialisation parameters for context identifiers ctx_cbp_luma, ctx_cbp_chroma 



Context 
label 


ctx_cb 


p_luma 




ctx_cbp_chroma 




ctx_cbp_chroma 


I si 


ices 


P,B 


slices 


Context 
label 


I slices 


P, B slices 


Context 
label 


I slices 


P, B slices 




m 


n 


m 


n 




m 


n 


m 


n 




m 


n 


m 


n 


63 


-3 


75 


-21 


81 


67 


-7 


65 


-23 


58 


71 


-18 


66 


-11 


46 


64 


-4 


63 


-15 


60 


68 


-I 


63 


-18 


64 


72 


-9 


67 


-6 


56 


65 


-A 


70 


-14 


61 


69 


-9 


77 


-16 


63 


73 


-13 


70 


-8 


59 


66 


-5 


56 


-15 


47 


70 


-4 


76 


-18 


73 


74 


-7 


74 


-18 


74 



Table 9-34 - Initialisation parameters for context identifiers ctx_cbp4, ctx_sig, ctxjast, ctx_abs_level for context 

category 0 — 4 





Co 
cate 


ntext 
gory 0 




Co 
cate 


ntext 
gory 1 




Context category 2 














Context 
label 


Context 
label 


Context 
label 


I slices 


P,B 
i slices 


Context 
label 


Context 
category 3 


Context 
label 


Context 
category 4 




m 


n 




m 


n 




m 


n 


m 


n 




m 


n 




m 


n 














ctx_cbp4 












75 


-4 


72 


79 


-4 


37 


83 


-2 


52 


-4 


57 


87 


0 


55 


91 


-3 


41 


76 


-4 


68 


80 


-3 


50. 


84 


-7 


68 


-9 


66 


88 


-3 


70 


92 


-4 


62 


77 


-6 


75 


81 


-6 


49 


85 


-5 


61 


-7 


64 


89 


-4 


68 


93 


-6 


58 


78 


-6 


75 


82 


-3 


6, 


86 


-8 


77 


-17 


80 


90 


-4 


75 


94 


-8 


73 














ctx_sig 












95 


-6 


68 








124 


-7 


67 


4 


46 


139 


-3 


71 








96 


-11 


66 


110 


0 


44 


125 


-11 


64 


1 


43 


140 


-9 


69 


142 


-6 


60 


97 


-5 


63 


111 


2 


53 


126 


-8 


66 


2 


45 


141 


0 


70 


143 


0 


63 


98 


-5 


56 


112 


0 


49 


127 


-11 


63 


-4 


46 








144 


-3 


54 


99 


2 


43 


113 


1 


43 


128 


-9 


63 


-1 


45 








145 


-4 


54 


100 




47 


114 


4 


45 


129 


-8 


60 


3 


43 








146 


4 


52 


101 


-8 


58 


115 


-2 


40 


130 


-II 


55 


-6 


44 








147 


-5 


44 



1 
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102 


-3 


46 


116 


-1 


45 


131 


-10 


61 


1 


45 








148 


-1 


48 


103 


4 


38 


117 


0 


50 


132 


-7 


63 


2 


46 








149 


-7 


57 


104 


0 


58 


118 


2 


55 


133 


-7 


60 


0 


46 








150 




51 


105 


1 


51 


119 


-7 


52 


134 


-16 


61 


-12 


49 








151 


-13 


51 


106 


-7 


57 


120 


-2 


57 


135 


-2 


62 


3 


50 




• 




152 


7 


55 


107 


-1 


53 


121 


7 


50 


136 


-1 


58 


11 


46 








153 


5 


57 


108 


2 


47 


122 


2 


52 


137 


-8 


61 


4 


50 








154 


2 


51 


109 


-1 


59 


123 


4 


66 


138 


-3 


68 


9 


64 








155 | 


4 


68 








ctx_last 






156 


0 


5 








185 


11 


25 


16 


27 


200 


12 


27 








157 


■ 


1 


171 


4 


42 


186 


9 


24 


21 


19 


201 


12 


28 


203 : 


10 


28 


158 


2 


2 


172 


5 


46 


187 


12 


24 


20 


23 


202 


16 


38 


204 


14 


30 


159 


3 


6 


173 


9 


40 


188 


14 


23 


21 


22 








205 


17 


30 


160 


4 


3 


174 


7 


41 


189 


13 


23 


21 


23 








206 


20 


30 


161 


5 


4 


175 


6 


46 


190 


16 


22 


24 


23 








207 


15 


37 


162 


6 


4 


176 


10 


40 * 


191 


19 


20 


25 


24 








208 


21 


39 


163 


7 


3 ; 


177 


14 


33 


192 


18 


21 


23 


27 








209 


22 


33 


164 


8 


4 


178 


10 


43 


193 


21 


21 


25 


29 








210 


21 


39 


165 


9 


5 


179 


12 


48 


194 


23 


25 


23 


35 








211 


15 


52 


166 


10 


9 


180 


13 


39 


195 


20 


23 


19 


36 








212 


8 


49 


167 


11 


2 


181 


13 


41 


196 


24 


25 


21 


40 








213 


13 


52 


168 


12 


3 


182 


16 


43 


197 


25 


29 


23 


45 








214 


8 


60 


169 


13 


1 


183 


21 


35 


198 


24 


33 


15 


53 








215 


15 


56 


170 


14 


6 


184 


11 


55 


199 


14 


53 


8 


70 








216 


3 


71 








ctx_abs_level 












217 


-5 


55 


227 


-10 


57 


237 


-10 


63 


-6 


51 


247 


-9 


70 


257 


-7 


58 


218 


♦3 


36 


228 


-1 


30 


238 


-5 


37 


-5 


24 


248 


-14 


55 


258 


0 


33 


219 


-1 


35 


229 


0 


32 


239 


-7 


43 


-7 


32 


249 


-10 


57 


259 


-1 


40 


220 


-2 


40 


230 


-1 


35 


240 


-6 


46 


-4 


34 


250 


-5 


56 


260 


-2 


45 


221 


-6 


50 


231 


0 


40 


241 


-5 


49 


-5 


39 


251 


-4 


57 


261 


-3 


49 


222 


•3 


44 


232 


-5 


39 


242 


-8 


50 


-12 


43 


252 


-14 


63 


262 


-7 


48 


223 


-A 


51 


233 


-4 


47 


243 


.7 


56 


-7 


50 


253 


-11 


67 


263 


-9 


58 


224 


-3 


53 


234 


-9 


55 


244 


-9 


62 


0 


48 


254 


-5 


68 


264 


-16 


66 


225 


-4 


55 


235 


-6 


58 


245 


-9 


64 


-3 


53 


255 


-9 


71 


265 


-12 


65 


226 


-11 


63 


236 


-4 


56 


246 


-11 


70 


-8 


60 


256 


0 


50 


266 


-12 


68 
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9.2.4 Table-based arithmetic coding 

^m!^ A ^ a i C ^ din8 If b .^ ed ^ P rinc, 'P' e of recursive interval subdivision. Given a probability estimation pCO') and 
p( 1 )=l-p( 0 of a binary decision ( 0 , *!'), an initially given interval with range R will be subdivided into two sub-intervis 
having ranged 0 )xR and R-p( 0 )xR, respectively. Depending on the decision, which has been observed, the corresponding sub- 
interval will be chosen as the new code interval, and a binary code string pointing into that interval will represent the sequence of 
observed binary dec.s.ons. It ,s useful to distinguish between the most probable, symbol (MPS) and the least probable symbol 
(LPS) so that bmary decisions have to be identified as either MPS or LPS, rather than '0' or «1\ Given this terminology^** 
context model CTX is defined by the probability of the LPS and the value of MPS, which is either '0' or « 1 '. ™ nOIOgy ' 
The arithmetic core engine in this Recommendation | International Standard has three distinct properties: 

" ^- P «° babmty es,imat ! on ls Performed by means of a finite-state machine with a table-based transition process between 
64 different representative probability states {P k 1 0£ k <64} for the LPS probability pu> s . 

- The range R representing the state of the coding engine is quantized to a small set {Q,....,Q4» of pre-defined 
quant.zaUon values pnor to the calculation of the new interval range. Storing a table containing all 64x4 pre-computed 
product values of QjX P k allows a multiplication-free approximation of the product Rx P k . 

- For syntax elements or parts thereof with an approximately uniform probability distribution a separate simplified 
encoding and decoding path is used. f«..«j 

9.2.4.2 Probability estimation 

The probability estimator is realized by a finite-state machine (FSM) consisting of a set of representative probabilities 
{P k I 0< k <64} for the LPS together with some appropriately defined state transition rules. Table 9-35 shows the 
transition rules for adapting to a given MPS or LPS decision. For transition from one state to another each state is only 
addressed by its index State, which will be appropriately changed to a new index Next_State_MPS(State) or 
Next_State_LPS(State) after the encoding/decoding of a MPS or LPS symbol, respectively. 

The numbering of the states is arranged in such a way that the state with index State=0 corresponds to a LPS probability 
value of 0.5, with decreasing LPS probability towards higher states. However, for I-slices it is of advantage to restrict 
the number of states to the first 24 state indices. Therefore, Table 9-35 contains a separate column containing the 
transition rule Next_State_MPS_INTRA that is used for decoding the syntax elements of an 1-slice only Note that 
Next_State_MPS_lNTRA differs from Next_State_MPS only by one entry. To prevent the FSM from switching to states 
higher than State=23 we set Next_State_MPS(35)= 23 for I-slice decoding. For the clarity of presentation, a separate 
table entry for I-slice decoding is shown in Table 9-35. v<"*™ 

After encoding or decoding a decision, an update of the probability estimate is obtained by switching the state index 
State to a new index, such that for I-slice coding 

if (decision = = MPS) 

State «- Next_State_MPS_INTRA (State) 

else 

State «- Next_State_LPS (State) 

and all other slice types 

if (decision = = MPS) 

State «- Next_State_MPS (State) 

else 

State <r- Next_State_LPS (State) . 

In the case, where the current state corresponds to a probability value of 0.5, which corresponds to the State index of 0 
and a LPS symbol is observed, the sense of MPS and LPS has to be interchanged. 



Table 9-35 - Probability transition 



State 


| Next_State_MPS_INTRA 


Next_Stare_MPS 


Next_State_LPS 


State 


| Next_State_MPS 


Next_State_LPS 


0 


1 


1 


0 


32 


33 


22 




2 


2 


0 


33 


34 


22 


2 


3 


3 


1 


34 


35 


23 


3 


4 


4 


2 


35 


36 


23 


4 


5 


5 


2 


36 


37 


24 


5 


6 


6 


3 


37 


38 


24 


6 


7 


7 


4 


38 


39 


25 
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7 


8 


8 


5 


39 


40 


25 


8 


9 


9 


6 ' 


40 


41 


26 


9 


10 


10 


1 


41 


42 


26 


10 


11 


11 


8 


42 


43 


27 


II 


12 


12 


8 


43 


i44 


27 


12 


13 


13 


10 \ 


44 j 


45 


28 


13 


14 


14 


10 


45 ; 


46 


28 


14 


15 


15 


10 ■ 


46 


47 


29 


15 


16 


16 


11 j 


47 


48 


29 


16 


17 


17 


12 


48 


49 


30 


17 


18 


18 


13 


49 


50 


30 


18 


19 


19 


14 


50 


51 


30 


19 


20 


20 


14 


51 


52 


31 


20 


21 


21 


14 


52 


53 


32 


21 


22 


22 


14 


53 


54 


33 


22 


23 


23 


15 


54 


55 


33 


23 


23 


24 


16 


55 


56 


34 


24 


-/- 


25 


17 


56 


57 


34 


25 


-/- 


26 


18 


57 


58 


35 


26 


-/- 


27 


19 


58 


59 


35 


27 


-/- 


28 


19 


59 


60 


36 


28 


-/- 


29 


20 


60 


61 


37 


29 


-/- 


30 


20 


61 


62 


37 


30 


-/- 


31 


21 


62 


63 


38 


31 


-/- 


32 


21 


63 


63 


38 



Table 9-36 - RTAB[State][Q] table for interval subdivision 



State 


0 


1 


2 


3 


State 


0 


1 


2 


3 


0 


9216 


11264 


13312 


15360 


32 


896 


1152 


1344 


1536 


1 


8832 


10816 


12800 


14720 


33 


896 


1088 


1280 


1472 


2 


8512 


10368 


12288 


14144 


34 


832 


1024 


1216 


1408 


3 


8128 


9920 


11712 


13504 


35 


832 


960 


1152 


1344 


4 


7680 


9344 


11072 


12736 


36 


768 


960 


1088 


1280 


5 


7168 


8768 


10368 


11968 


37 


768 


896 


1088 


1216 


6 


6912 


8448 


9984 


11520 


38 


704 


896 


1024 


1152 


7 


6336 


7808 


9216 


10624 


39 


704 


832 


960 


1152 
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8 


5888 


7232 


8512 


7oJO 


40 


640 


832 


960 


1088 


9 


5440 


6656 


7 877 
/ o /Z 




41 


640 


768 


896 


1088 


10 


5120 


6208 


7360 


oJ 1 Z 


42 


640 


768 


896 


1024 


1 1 


4608 


5632 


6656 


7680 


43 


576 


704 


832 


960 


12 


4224 


5184 


6144 


71fl4 


«f4 


576 


704 


832 


.. 960 


13 


I 3968 


4800 


5696 


O.J>Z 


45 


576 


704 


832 


960 


14 


3712 


4480 


5312 


AlAA 


4o 


512 


640 


768 


896 


15 


3456 


4224 


4992 




4/ 


512 


640 


768 


896 


16 


3072 


3776 


4416 


JIZU 


45 


512 


640 


768 


832 


17 


2816 


3456 


4096 


47"V» 

*+ / JU 




512 


640 


704 


832 


18 


2624 


3200 


3776 


ID 


1 CA 

50 1 


512 


576 


704 


832 


19 


2432 


3008 






51 


448 


576 


704 


768 


20 


2304 


2816 


3328 


■IJMft 


JZ 


448 


576 


640 


768 


21 


2048 


2496 


2944 


jjyz 


53 


448 


512 


640 


704 


22 


1856 


2240 


7/*88 


JU/z 


54 


448 


512 


640 


704 


23 


1664 




Z*K>Z 


2816 


55 


448 


512 


576 


704 


24 


1536 


J oJQ 


zz**u 


2560 


56 


384 


512 


576 


704 


25 


1408 


1728 




2368 


57 


384 


512 


576 


640 


26 


1344 


1600 


1 Q7fi 


21 /O 


58 


384 


448 


576 


640 


27 


1216 


1472 


1 709 
J / yz 


ZlWo 


59 


384 


448 


576 


640 


28 


1 152 


1408 


1 Uv>H 


t OOA 1 

iyzu i 


60 


384 


448 


512 


640 


29 


1088 


1344 


1536 


1792 | 


61 


384 


448 


512 


640 


30 


1024 


1280 


1472 


1728 | 


62 


384 


448 


512 


576 




960 


1216 


1408 


1600 | 


63 


384 


448 


512 


576 



9.2.4.3 Description of the arithmetic decoding engine 

The status of the arithmetic decoding engine is represented by a value V pointing into the code sub-interval and the 
corresponding range R of that sub-interval. Figure 9-3 gives an illustration of the whole decoding process. Performing 
the InitDecoder procedure, which is further specified in subclause 9.2.4.3.1, appropriately initialises V and R For 
decoding of each single decision 5, the following two-step operation is employed: First, the related context model CTX is 
determinedaccording to the niles specified in subclauses 9.2.2. Given the context model CTX, the decoding operation 
Decode(C7X) then delivers the decoded symbol S as is described in detail in subclause 9 2 4 3 2 
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I nit Decoder 



Read CTX 

No 



S = Encode (CTX) 




Yes 

( °~ ) 

Figure 9-3 - Overview of the Decoding Process 



9.2.43.1 Initialisation of the decoding engine 

In th initialisation procedure of the decoder, as illustrated in Figure 9-4, V is first filled with two bytes of the 
compressed data using the GetByte routine as specified in subclause 9.2.4.3.4, and then the range R is set to 0x8000. 
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^ Done 

Figure 9-4 - Flowchart of initialisation of the decoding engine 



9*2.4^.2 Decoding a decision 

l l8U ^t Sh ° WS fl ° wchart f° r deeding a single decision. In a first step, the estimation of the sub-interval ranges 
Rlps and R MPS corresponding to the LPS and the MPS decision is performed as follows. 

Given the interval range /?, we first map R to a quantized value Q using 

Q=(R-Ox4001)»12, 



such that the state index State and Q are used as an entry in the look-up table RTAB to determine R, 

Rws=RTAB[ State JfQJ. 



LPS- 



(9-11) 



(9-12) 



Jnzttl 6 , SPeCifl l S COrres P on f n e v /' ues °f ^AB in 16-bit representation. The tabulated values are actually given 
in 8-bit accuracy; the maximum value of RTAB corresponds to 14 bits and all values have been left-shifted by 6 bite for 
a better access in a 16-bit architecture. J 

In a second step the current value of V is compared to the size of the MPS sub-interval R MPS . If V is greater than or equal 
h™ o decoded, V ,s decremented by R MPS and the new range R is set to R^; otherwise a MPS is decoded and 
t^^Sr k t^TTifi? *"? GiVe " ^ deCiSi ° n ' the P robabi »ty update is performed accordingly 

as !S '•" Tf •> A \ ? epe "?^ °" ^ CUmat Va,UC ° f the new ran « e * -normalization ™H be performed 
as described in more detail in subclause 9.2.4.3.3. 
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^Decode (CTX) 



Q = (R-Ox4001)»12 
r lps = RTAB[State<CTX)3lQ3 

^MPS = " " ^LPS 




Yes 



MPS(CTX) = MPS(CTX)*1 



State<CTX) = Next_State_LPS{State(CTX)] 



RenormD 



^ Pone 

Figure 9-5 - Flowchart for decoding a decision 



9.2.4.33 Renormalization in the decoding engine (RenormD) 

Renormalization is illustrated in Figure 9-6. The current range R is first logically compared to 0x4000: If it is greater 
than that value no renormalization is needed and RenormD is finished; otherwise the renormalization loop is entered. 
Within this loop, the range R is doubled, i.e. left-shifted by 1 and the bit-counter BG is decremented by 1. In the case, 
that the condition BG<0 holds, a new byte of compressed is inserted into B by calling the GetByte routine. Finally, the 
next bit of B is shifted into V. 
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Figure 9-6 - Flowchart of renormaJization 



9.2.43.4 Input of compressed bytes (GetByte) 

Figure 9-7 shows how the input of compressed data is performed. At the initialsation stage of the whole decoding 
PS"? new hT* f ren ° rmal, ^° n ° CCUrS / nd *» EG has a negative value, this procedure wiH be 

First a new byte of compressed data ,s read out of the bitstream C; then the index CL pointing to the current poskion of 
the bitstream C is incremented by 1 and the bit-counter is set to 7. mn-em position or 



^ Get Byte ^ 



B = CfCL] 
CL = CL ♦ 1 
BG = 7 



Figure 9-7 - Flowchart for Input of Compressed Bytes 



9.2.4.3.5 Decoder bypass for decisions with uniform pdf (Decode_eq_pr b) 

This special decoding routine applies to decoding of the sign information of motion vector data and the sign of the levels 
of s.gn.ficant transform coefficients, which are assumed to have a uniform probability distribufion Conseouentlv 
omut.ng the probab, hty est.mation in this special case reduces the decoding process to a single comparison 

l 0 °±r t0 ,- etermine the SUbimerVal 3nd kS «™*™*o* decoded symbol v'lueT Z l lZeau^t 

renormal.zat.on process .s s.m.lar to that performed in the renormalization procedure RenormD. as depicted in RgX 9 
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8 with two modifications. Firstly, the rescaling operation R«R«1) is unnecessary and secondly, the initial comparison 
(R<=0x4000?) can be omitted. 




V = (V«1) | (B&1) 
B = B»1 



Done ^ 



Figure 9-8 - Flowchart of decoding bypass 



10 Decoding process for B slices 
10.1 Introduction 

The use of B (bi-predictive) slices is indicated in the naLunit.type. A B slice is an inter predicted slice The major 
difference between a B slice and P slice is that B slices are coded in a manner in which some macroblocks or blocks may 
use a weighted average of two distinct inter prediction values for building the prediction signal. Generally, B slices 
utilize two distinct reference index lists. Each of these index lists refer to pictures in the reference picture buffer. 
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Figure 10-1 - Illustration of B picture concept 



^V-.? C loca " on ° f p,ctures ,n *e brtstream is in a decoding order. Pictures that are dependent on other pictures shall occur 
in the ^stream after the pictures on which they depend. Figure 10-1 shows one hypothetical example, where ^Tpictur^Te 
inserted in output order between an I and a P picture and the subscripts indicate the output order. The P picture P 4 onfy depenl on 
the first Intra p.cnire I 0 . The B picture B 2 , which is temporally located between I 0 and P 4 , depends on bou, of the4 Sr^e B 

10.2 Decoding process for macroblock types and sub macroblock types 

There are five different prediction modes supported by B pictures. They are the list 0, list 1, bi-predictive, direct and 
intra prediction modes. In bi-predictive mode, the prediction signal is formed by a weighted average of list 0 and list 1 
prediction values. The direct mode can result in prediction modes list 0, list 1, or bi-predictive. Prediction using the 
direct mode is derived from a combination of the motion vectors and macroblock type used in the co-located macroblock 
of the first picture (the picture at index 0) in list 1 . «n«u.rooiocK 

Coded macroblocks in B pictures utilize a similar tree-structured macroblock partitioning to P pictures. Depending on the 
number of elements in the two reference picture lists, up to two reference picture indices are coded for each bi-predicted 
re8 '° n K: Additionally for each luma block of 16x16, 16x8, 8x16 samples and the associated chroma blocks, and each sub 
macroblock the prediction mode (list 0, list 1, bi-predictive, direct, intra) can be chosen separately. In order to avoid a 
separate codeword to specify the prediction mode.the indication of the prediction direction is incorporated into the 
codewords for macroblock type and sub macroblock type, respectively, as shown in the Table 7-12 and Table 7-17 A 
sub macroblock of a B picture macroblock can also be coded in direct mode. 

When mb adaptive_frame_field_fla g == 1, the current direct-mode macroblock shall follow the same frame/field 
coding mode as its co-located macroblock. 



103 Decoding process for motion vectors 
103.1 Differential motion vectors 

In the following, the terms temporal ordering and temporal distance refer to ordering and distance according to the 
picture order counter described in subclause 8.3.2. This is also the decoder output order and the intended display order. 

Motion vectors for list 0, list 1, or bi-predictive blocks are differentially encoded. A prediction has to be added to the 
motion vector differences in order to reconstruct motion vectors for the current macroblock. 

As a special case of bi-predictive blocks, if the two reference pictures used for the bi-prediction both occur temporally 
earlier or both occur temporally later than the current picture being decoded, then the motion vector decoding is 
performed as described in subclause 10.3.2. B 

Otherwise, the predictions for bi-predictive blocks are formed from the motion vectors of spatially neighbouring blocks 
in a way similar to that described in subclause 8.4. 1 , but with a few important distinctions. 

First, a list i, where i = 0 or 1, motion vector MV, from the current block is predicted only from neighbouring blocks that 
contain mot.on vectors with the same temporal direction (earlier or later in time) as MVi. If a neighbouring block does 
not have a motion vector with the same temporal direction, the predictor for that block is set to zero and the neighbouring 
ofTubclauL 8 4°rT ^ bel ° nging l ° 3 different reference P icture for P ur P° ses of computing the median prediction 

Second if a neighbouring bi-predicted block has both motion vectors pointing in the same temporal direction as MV 
and both motion vectors point to the same reference picture, then the list 0 motion vector from that block is used as a 
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prediction Otherwise, if a neighbouring bi-predicted block has both motion vectors pointing in the same temporal 
direction as MV-, but they point to different reference pictures, then the motion vector that points to the temporally closest 
reference picture is used. 

Third, reconstructed motion vectors in direct mode neighbouring blocks shall be used as predictions for the current block 
motion vectors. 

If a direct mode neighbouring block has two motion vectors, then this block is treated as if it were a bi- 
predictiven ighbouring block. 

If a direct mode neighbouring block has only one motion vector, then this block is considered as a list 0 or list 1 
block. 

In the case that the co-located block in such a direct mode block is intra coded and the 
direct_spatial_mv_pred_flag is 0, the direct mode block is treated as belonging to a different reference picture 
for purposes of computing the median prediction of subclause 8.4.1.1. 

10.3.2 Motion vector decoding with scaled M V 

If both reference pictures (ref.idxJO & ref_idx_ll) occur temporally earlier or both occur temporally later than the 
current picture being decoded, then the following process is followed for decoding the motion vectors MV } and MV 2 for 
bi-predictive modes. The motion vector decoding process is illustrated in Figure 10-2. The motion vector Af V, for the 
first reference picture (refJdxJO) is differentially decoded using motion vector prediction as described in 10.3.1. 
However, the method used for decoding the motion vector MV 2 for the second reference picture (ref.idxjl) is as 
follows: 

The scaled motion vector (smv) is first calculated from the motion vector M V, as: 

Z = (TD 2 x 256) / TD X smv = (Zx M V, + 128) » 8 

where TD X is the temporal distance between the current picture and the reference picture indicated by refjdxJO, and 
TD 2 is the temporal distance between the current picture and the reference picture indicated by refjdxjl. 

Then, MV 2 is differentially coded with respect to smv. 
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Figure 10-2 - Differential motion vector decoding with scaled motion vector 



10.3.3 Motion vectors in direct mode 

The direct_spatial_mv_pred_flag identifies for the current slice whether the direct mode motion vectors are calculated 
using a spatial or temporal technique. If this indicator is set to 1 then the spatial technique is used. Otherwise, if this 
indicator is set to 0, then direct mode motion vectors are calculated using the temporal technique. 

10.33.1 Spatial technique of obtaining the direct mode motion parameters 

The first step in the spatial technique for direct mode prediction is the determination of a candidate reference picture 
index for each list (list 0 and list 1). 

The reference picture indices for the neighbouring blocks A, B, C, and D within the current slice for the 16x16 current 
luma block E, as described in subclause 8.4.1.1 and shown in Figure 8-4, shall be used to determine a preliminary 
candidate reference picture index for each list. The preliminary candidate reference picture index for each list (list 0 and 
list 1) shall be the minimum reference picture index among the reference picture indices used from the same list for the 
prediction or bi-prediction of the neighbouring blocks. If no neighbouring blocks are present within the current slice that 
use prediction from the same list, either for prediction or bi-prediction, the preliminary candidate reference index for that 
list shall be interpreted as not existing. 

If a preliminary candidate reference index exists for either list, decision of the final candidate reference indices and 
associated motion vector values for that list for each 4x4 block of the current macroblock depends on the coded 
parameters of the co-located 4x4 blocks in the first picture in list 1 . If the first picture in list 1 is a short-term picture and 
if all lines of the co-located 4x4 block were predicted using list 0 prediction with reference picture index 0 and motion 
vector components in the range of -1 to 1 inclusive, then the final candidate reference picture index for the list of the 
current 4x4 block shall be 0 and shall be associated with a candidate motion vector value of (0, 0). Otherwise, the final 
candidate reference picture index for the list shall be the preliminary candidate reference picture index for the list and the 
associated candidate motion vector shall be obtained, using the 16x16 block motion vector prediction, as described in 
subclause 8.4.1 by using the final reference picture index. 

If both candidate reference picture indices exist, then the block is predicted as a bi-prediction block using the final 
candidate reference picture index and motion vector for each list. Otherwise, if a candidate reference picture index exists 
for only one of the two lists, the block shall be predicted by single-list prediction using the final candidate reference 
picture index and associated motion vector for the existing candidate. Finally, if neither candidate reference picture index 
exists, bi-prediction shall be used with reference picture index zero and associated motion vector (0, 0) for both lists. 

10.3.3.2 Temporal technique of obtaining the direct mode motion parameters 

In the temporal technique direct mode, the same block structure as for the co-located macroblock of the first picture (the 
picture at index 0) in list 1 is used. For each block of the current macroblock, the list 0 and list 1 motion vectors are 
computed as scaled versions of the list 0 motion vector of the co-located block in the list 1 reference picture as described 
below. 



NSOOCID: <XP 1 100641A_1_> 



ITU-T Rec. H.264 (2002 E) 



117 



ISO/IEC FCD 14496-10 



The list 0 reference picture for the direct mode current block is the same as the list 0 reference picture used for the co- 
located block in the list 1 reference picture. The list 0 and list 1 motion vectors for direct mode macroblocks are 
calculated differently depending on whether picture_struct and the reference indicate fields or frames. Also if the list 1 
co-located macroblock is an intra-coded block, the motion vectors are set to zero, and the list 0 reference picture for the 
direct mode is the most recent temporally preceding stored picture. 

With possible adaptive switching of frame/field coding at picture level, a B frame or its list 1 reference frame can be 
coded in either frame structure or field structure. Hence, there are four possible combinations of frame or field coding for 
a pair of a macroblock in the current B picture and its co-located macroblock in the list 1 reference picture. Calculations 
of the two motion vectors in direct mode are slightly different for the four cases. 

Case 1: Both the current macroblock and its co-located in the list 1 reference picture are in frame mode, as shown in 
Figure 10-3. The list 0 reference for each block within the current macroblock is the same as the list 0 reference of the 
co-located block in the list 1 reference picture. Two motion vectors (AfV F , MV B ) are calculated by 

Z = (TD B x 256V TD D MV F = (Zx MV + 128) » 8 

W=Z- 256 M V B = (Wx MV + 128) » 8 

where TD B is the temporal distance between the current B frame and the list 0 reference frame, and TD D is the temporal 
distance between the list 1 reference frame and the list 0 reference frame (see Figure 10-3). 

In the case that the co-located block in the list 1 reference frame has a list 0 motion vector pointing to a long-term frame, 
the list 0 and list 1 motion vectors for the direct mode current block are calculated by 

MV F =MV 

MV B = 0 ' 



List 0 Reference Current B List 1 Reference 

fO ft K> fl fO f1 




Figure 10-3 - Both the current block and its co-located block in the list 1 reference picture are in frame mode (fO 
and fl indicate the corresponding fields) 



Case 2: Both the current macroblock and its co-located macroblock in the list 1 reference picture are in field mode. Two 
motion vectors for each block within the current macroblock are calculated from the list 0 motion vector of the co- 
located block in the list 1 reference field of the same parity. 

For field 0, the list 0 motion vector of the co-located block will always point to one of the previously coded list 0 fields, 
as shown in Figure 10-4. The list 0 reference field for the direct mode block will be the same as the list 0 field of the co- 
located list 1 block, and the list 1 reference field will be field 0 of the list 1 reference frame. The list 0 and list 1 motion 
vectors {MV Ftb MV B .d for the direct mode block are calculated as follows: 

Z= 77V. X 256/ rZ) a , MV FJ = (ZxMVi +128) » 8 
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W=Z-256 



MV B/t = (Wx A/V, +128) » 8 



where the subscript i is the field index (0 for the 1st field and 1 for the 2 nd field), and MV % is the list 0 motion vector of 
the co-located block in field i of the list 1 reference frame. TD Bi is the temporal distance between the current B field and 
the list 0 reference field. TD DJ is the temporal distance between the list 1 reference field and the list 0 reference field. 

For field 1, the list 0 motion vector of the co-located list 1 block may point to one of the temporally previous coded 
fields, in which case calculation of the list 0 and list 1 motion vectors follows the same equations as above. 

However, it is also possible that the list 0 motion vector of the co-located block in field 1 of the fist 1 reference frame 
points to field 0 of the same frame, as shown in Figure 10-5. In this case, the two motion vectors for field 1 of the direct 
mode current block are calculated as follows: 



Z= -TD BA x 256/ TD DJ MV fa = (Zx MV V +128) » 8 



W=Z-256 



MV BA = (Wx MV X +128) » 8 



Note that both motion vectors are now pointing to field 0 and field 1 of the list 1 reference frame respectively. 
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Figure 10-4 - Both the current macroblock and its co-located macroblock in the temporally subsequent picture 

are in field mode. 



In the case that the co-located block in the list 1 reference field has a list 0 motion vector pointing to a long-term field, 
the list 0 and list 1 motion vectors for the direct mode are calculated by 

M V F j = MVj 



MV BJ =0 
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currant block 



co^ocated block 




Figure 10-5 - The list 0 motion vector of the co-located block in field 1 of the list 1 reference frame may point t 
field 0 of the same frame. 



Case 3: The current macroblock is in field mode and its co-located macroblock in the list 1 reference picture is in frame 
mode, as shown in Figure 10-6. Let y^re* and y co .,oca t ed be the vertical indices of the blocks in the current macroblock and 
in its co-located macroblock respectively. Then, y co -iocated = 2 x y CMrrcn ,. The blocks in the current macroblock and its co- 
located macroblock have the same horizontal indices. The list 0 reference field is the same-parity field of the list 0 frame, 
and the list 1 reference field will be the same-parity field of the list 1 reference frame as shown in Figure 10-6. 




Time 



Figure 10-6 - The current macroblock is in field mode and its co-located macroblock in the list 1 reference picture 
is in frame mode 



The motion vectors of a direct mode block are calculated from the list 0 motion vector of the co-located block in the list 
1 reference frame as follows: 

Z = TD B jX 256/ TD D MV FA = (Zx MV +128) » 8 
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W=Z-256 MV B , t =(WxMV +128) » 8 



In the case that the block in the list 1 reference frame used for the direct mode motion vector calculation has a list 0 
motion vector pointing to a long-term picture, the list 0 and list 1 motion vectors for the direct mode are calculated by 

MV FJ = MV 

MV Bi = 0 

Case 4: The current macroblock is in frame mode while its co-located macroblock in the list 1 reference picture is in Field 
mode, as shown in Figure 10-7. Let y cumm and y eo . locattd be the vertical indices of the blocks in the current macroblock 
and in its co-located macroblock respectively. Then, y ahlataM = y Cttmal I 2. The blocks in the current macroblock and in 
its co-located macroblock have the same horizontal indices. The two fields of the co-located block in the list 1 reference 
may be coded in different modes. Since field 0 of the list 1 reference is temporally close to the current B picture it is 
used in calculating the motion vectors and determining the references for direct mode current blocks as shown in Figure 

Two frame-based motion vectors of direct mode block are calculated as follows: 



Z^TD B x 256/ TD D0 



MV F = (Zx MV 0 +128) » 8 



W=Z-256 



MV B = (Wx MV 0 +128) » 8 




Time 



Figure 10-7 - The current macroblock is in frame mode while its co-located macroblock in the list 1 reference 
picture is in field mode. 



In the case that the block in the list I reference field used for the direct mode motion vector calculation has a list 0 
motion vector pointing to a long-term picture, the list 0 and list 1 motion vectors for the direct mode are calculated by 

MV F -MV 0 

MV B =0 

Ed.Note: I don't understand the following sentence. More explanation maybe needed here. This appears to indicate that 
if the co-located macroblock is in field, then also field mode should be used for the current macroblock/block. 
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Considering though that AFF is done in pairs of macroblocks, is somehow the whole group forced to be in the same 
mode as its co-located? This is what this seems to indicate.] When mb_frame_field_adaptive_flag == 1, the current 
direct-mode macroblock is in the same frame/field coding mode as its co-located macroblock. 

10.4 Weighted prediction signal generation procedure 

If weighted_pred_ flag is equal to one, explicit weighted prediction is applied to P and SP slices. If 
weighted_pred_explicit_flag is equal to one, explicit weighted bi -prediction; is applied to B slices. If 
weighted jMpred_implicit_f!ag is equal to one, implicit weighted bi-prediction is applied to B slices. 

10.4-1 Weighted prediction in P and SP slices 

In P and SP slices, when weighted_pred_flag is equal to one, weighted prediction is applied for predicted macroblocks. 
When 0 < mb_type < 4, the luma prediction is generated as 

P = clipl((F 0 x W 0 + 2 s "** ) » LWD+ O 0 ) 

and the chroma prediction block is generated as 

P = clip 1 (128 + «CP 0 - 128) x CW 0 + 2 0 ™" 1 ) » CWD+ CO 0 ) 

where 

F 0 = reference prediction block v 
LWD - luma_Jog_weight_denom 
W 0 = luma_weightJO[ref JdxJO] 
O 0 = luma^offset_10(reLidx_10] 
j = 0 for Cb and 1 for Cr 
CP Q - chroma reference prediction block 
CWD = chroma_log_weight_denom 
CW 0 = chroma_weight_10[ref_idx_10][j] 
CO 0 = chroma_offsetJ0[ref_idx_10][j] 
To limit the calculation to 16-bit precision, the following conditions shall be met: 

-128<W 0 <127 
-128<CW 0 <127 
10.4.2 Explicit weighted bi-prediction in B slices 

In B slices, when weighted J)ipnri_explicit_flag is equal to one, weighted prediction is applied for predicted 
macroblocks. For reference list 0 prediction, the luma prediction is generated as 

P = clipl((P 0 x W Q + 2 LWD ' A ) » LWEhr O 0 ) 

For reference list 1 prediction, the luma prediction block is generated as: 
P = clipl ((/> X W x + 2 LW °- 1 ) » LWD+ O, ) 

where 

P 0 = reference prediction block from list 0 
P x = reference prediction block from list 1 
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LWD = luma_log_weight_denom 
W 0 = luma_weightJO[ref Jdx JO) 
W t = luma_weightj 1 [ref Jdx J 1 ] 
O 0 = Iuma_offset JOfrefjdx JO] 
0, = Iuma_offsetJl[refJdxJl] 
When bi-prediction is used, the luma prediction block is generated as: 

P = clipl ((/> x *ZW 0 + /> x Z?Z)^ + 2 L ™) » (LWD + 1^ BDO) 

where 

£Z>Wb = Iuma_weight_bipred_IO[ref_idxJO][ref_idxJl] 
£Z>W, = luma_weight_bipred_ll[ref_idx_IO][reL.idx_Jl] 
£Z><9 = luma_ofTsetJ>ipred[refJdxJO][refJdxJl] 

For reference list 0 prediction, the chroma prediction block is generated as 

P = clipl(l 28 + ((CP 0 - 128) x CW 0 + 2 CTO - 1 ) » CO 0 ) . 

For reference list 1 prediction, the chroma prediction block is generated as: 

P = clipl (128 + ((CP, - 128) x CW, + 2 aw> - 1 ) » CWZH CO t ) . 
When bi-prediction is used, the chroma prediction block is generated as 

P = clipl (128 + ((CP 0 - 128) x CBDW 0 + (OJ - 1 28) x CBDWJ + 2*™) » (C WD + l>f CZ?D<9) 

where 

y = 0forCband 1 forCr 

CP Q = chroma reference prediction block from list 0 

CP t = chroma reference prediction block from list 1 

CWD - chroma Jog_weight_denom 

CW 0 = chroma_weight JOfrefjdx J0)[j ] 

CW X = chroma_ weight Jl [ref JdxJ l][j ] 

CO o = chroma_offsetJ0[reMdxJ0][ j ] 

CO , = chroma_offsetJl[refJdxJl][ j ] 

CBDW 0 = chroma_weight_bipredJO[refJdxJO][refJdxJl]|JJ 
CBDW , = chroma_weight_bipred_ll[refJdxJO][refJdx_ll](j] 
CBDO = chroma_offset_bipredfref_idx_10][ref_idxjl]lj] 
To limit the calculation to 16-bit precision, the following conditions shall be met: 

-128<W 0 <127 
-128<W; <127 
-128<H/ 0 +W f <127 
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-128<CW 0 <127 
-128 <CW,< 127 
- 128 < CW 0 + CW, < 127 
- 128 < BDW 0 + BDW l < 127 
- 128 < CBDW 0 + CBDW l < 127 
10.4.3 Implicit bi-predictive weighting 

When weighted_bipred_implicit_flag is equal to 1, the prediction weighting factors are not sent explicitly and the luma 
and chroma predictions are generated as follows. 

If the decoding order of the reference picture indicated by ref_idx_10 is previous to or the same as that indicated by 
ref_idx_ll, or for skipped macroblocks or direct mode, the prediction signals are generated as follows: 

,/>=clipl(/ > 0 x2-/i) • 

where 

P 0 = reference prediction block from list 0 
Pi = reference prediction block from list 1; 
otherwise, the prediction signals are generated as follows 

P = (P 0 + i>+l)»l 

where 

P 0 = reference prediction block from list 0 
/>! = reference prediction block from list 1 . 

11 Decoding process for SP and SI slices 
11.1 General 

SP slices make use of motion-compensated predictive coding to exploit temporal redundancy in the sequence in a 
similar manner to P slices. SI slices make use of spatial prediction, in a similar manner to I slices. Unlike P slices, 
however, SP slice coding allows identical reconstruction of a slice even when different reference pictures are being used. 
SI slice coding allows identical reconstruction to a corresponding SP slice. 

NOTE - Above mentioned properties of SP and SI slices provide functionalities for bitstream switching, splicing, random access, 

VCR functionalities such as fast-forward, and error resilience/recovery. 
An SP slice consists of macroblocks coded either as Intra type (Intra_4x4 or Intxa_l6xl6) or Pred type (MbSkip 
Pred_L0_16xl6, Pred J-0LCL1 6x8, Pred_L0_L0_8x 1 6, Pred_8x8 or Pred_8x8refO). An SI slice consists of 
macroblocks coded either as Intra type or SIntra4x4 type. 

Intra macroblocks in SP slices shall be decoded as described in subclause 8.3. All other macroblocks, including MbSkip, 
are decoded as described below. 

The Intra_8x8 sub-partition mode shall not be present in SP slices. 
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Intra macroblocks in SI slices are decoded as described in subclause 8.3, with the addition that the prediction mode of a 
neighbouring SIntra_4x4 block is considered to be "mode 2: DC prediction". SIntra_4x4 macroblocks are decoded as 
described below. 



11.2 SP decoding process for non-switching pictures 

This subclause applies to all macroblocks in SP slices in which sp_for_switch_flag = = 0, except those with slice typeO 
equal to Intra_4x4 or Intra_16xl 6. It does not apply to SI slices. 

Figure 11-1 depicts generic decoding for non-intra coded macroblocks in SP slices. The prediction P(x,y) for the current 
macroblock of the slice being decoded is formed by the motion compensated prediction block using the same process as 
is used in P slice decoding. After forming the predicted block P(x,y), decoding is performed as follows. 
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Figure 11-1 - A block diagram of a conceptual decoder for non-intra coded macroblocks in SP slices in which 

sp_for_switch_flag = = 0. 



11.2.1 



Luma transform coefficient decoding 



The predicted block, P(x,y) f where P(x,y) ={ Poo ... p 33 }, is transformed according to Equation 1 1-1 to produce transform 
coefficients c . 
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The received prediction residual coefficients, w 21 **, are scaled using quantisation parameter QP, and added to the 
transform coefficients of the prediction block, as in Equation 11-2. 



c/ R,OR = Cij PRED + (((wg« *R,«""» *A y ) « (QP/6)) » 6) i, j = 0.....3 



QERR *o (QP%6) 



(H-2) 



where R is defined in Equation 8-40, and where A is defined as: 
V"= 16 for(ij)= {(0,0),(0,2),(2,0),(2,2)}, 
Aj*> = 25 for (i,j) = {(1,1),(1,3),(3,1),(3,3)}, 
Aij m) = 20 otherwise;. 

For luma, QP = QP y , as defined in Equation 7-7 and 7-9. 

The coefficients &/ m; , used in the formulas below, are defined as: 
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Qj m) = for (i j) = {(0,0),(0,2),(2,0),(2,2)}, 
Qj"° = VL.i for(ij)= {(1,1),(1,3),(3,1).(33)J, 
Qij m) = M m , 2 otherwise; 

where the first and second subscripts of M are row and column indices, respectively, of the matrix defined as: 



M = 



The resulting sum, c PRIOR , is quantised with a quantisation parameter QS, as in Equation 1 1-3. 

For luma, QS = QS r , which is defined in Equation 7-8. 

Cij QREC = {Sign(c ij PR,OR )*[Abs(c ij PR,OR )*Q ij (QS * 6) + (l«(15+QS/6))]}»(l6+QS/6) 

i,j=0,...,3 



"13107 


5243 


8066 


11916 


4660 


7490 


.10082 


4194 


6554 


9362 


3647 


5825 


8192 


3355 


5243 


7282 


2893 


4559 



(11-3) 



These quantised levels, c QREC , are scaled as in Equation 1 1-4. 

Wij QREC = ( Cij QREC * Rg (QS% ?) « (QS/6) i, j - 0,. . 3 



(11-4) 



where R is defined in Equation 8-40. 

The transform and reconstruction processes are performed for these scaled levels, as defined in Equations 8-48 through 
8-59. Finally, after the reconstruction of a macroblock, filtering takes place as described in subclause 8.7. 

11.2.2 Chroma transform coefficient decoding 

The decoding of chroma components for non-intra coded macroblocks in SP slices is similar to the decoding of luma 
components. 

The predicted block, P(x,y), where P(x,y) = {poo - P33}, is transformed according to Equation 11-1 to produce transform 
coefficients c PRED . An additional 2x2 transform is applied to the DC coefficients of these blocks. The 2 dimensional 2x2 
transform procedure is defined in Equation 11-5: 



PRED 



[1 iJOCoo DC 0l Tl 1] 
LI DC U \\ -lj 



(11-5) 



The received DC prediction residual coefficients, w 0 ™, are scaled using quantisation parameter QP, and added to the 
DC transform coefficients of the prediction block, as in Equation 1 1-6. 

c/* 10 * = Cij PRED + «(w y QERR * Ri/^ * A y ) « (QP/6)) »5) i, j =0,..., 3 (1 1-6) 

The resulting sum, c^ 10 *, is quantised with a quantisation parameter QS, as in Equation 1 1-7. 

Cy QREC = {Sign^^nAbs^/^^Qii^ + (l«(16+QS/6))]}»(17+QS/6) 



i,j = 0, 3 



(11-7) 



These quantised levels, Ci/ 31 ^, are scaled as in Equation 1 1 -4. 

AC coefficients are decoded in an identical process to mat used for luma. 

The value of QP to be used for chroma data, denoted QP& is obtained from QP Y using the relationship specified in Table 
9-1. Similarly, the value of QS to be used for chroma data, denoted QS& is obtained from QS Y using the relationship 
specified in Table 9- 1. 
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11.3 SP and SI slice decoding process for switching pictures 

This subclause applies to all macroblocks in SP slices in which s P _for_switch_flag = = 1, except those with slice tvoeH 
TlnJa M6 " Intni - 16X,6: "* l ° *" macrob,ocks in SI except those with sliceJypeO eq3\o ,£32 

Figure 11-2 depicts generic decoding for macroblocks in SI and SP slices that are not Intra coded In SP slices the 
predion P(x,y) for the current coded macroblock of the slice being decoded is formed by the inter prediction block 
using the same process as is used m P slice decoding. In SI slices, the prediction P(x,y) is formed by intra prediction 
usmg uhe same process as is used in I slice decoding. After forming the predicted block P(x,y), the decoding of SI and 
non-intra coded macroblocks in SP slices follows the same steps. 8 d 



from 

encoder 
► 



Inverse j 
Quantisation!^ 


Inverse 






Transform 


> 


Loopfilter 



decoded 
video 



Quantisation 



Transform 



P(x,y) ^1 MC 

prediction 



Intra 
prediction 



frame 
mem ory 



R(x,y) 



motion information 



Intra prediction mode infotmation 



Figure 11-2 - A block diagram of a conceptual decoder for non-intra macroblocks in SI slices; and for non-intra 
coded macroblocks in SP slices in which sp_for_switch_flag = = 1. 



When decoding SIntra_4x4 macroblocks, the intra prediction modes of the neighbouring SIntra 4x4 and Intra 4x4 
blocks are taken into account as described in subclause 8.5. ^" intra_4x4 

When decoding Intra_4x4 macroblocks, the intra prediction modes of the neighbouring Intra 4x4 blocks are taken into 

i^z:Ts i ^:^ lause 8 - 5, but prediction mode o(the neighbouring si *«*- 4 * 4 

113.1 Luma transform coefficient decoding 

Joe7ficS2^^£f' W r here ^ =iPW "■ 55 transformed accordi «g t° Equation 11-1 to produce transform 
coefficients c . These transform coefficients are then quantised with a quantisation parameter QS, as in Equation 1 1-8. 

__pred _ (Signfe/^nAbsfe/^^Qy^)-, (l«(15 + QS/6))]}»(16 + QS/6) i, j = 0 
3 (11-8) 

Note: Equation 1 1 -8 is the same as Equation 1 1 -3 except for the change of name of input and output variables. 
For luma, QS = QS r , which is defined in Equation 7-8. 

U^^^ZT*™ 1 COefficientS ' « added «° th - quantised transform coefficients of the prediction 

Cij Q RE c = Wy r RED + WijQERR iJ = 0 ,..., 3 (u . 9) 

These quantised levels, c QREC t are decoded as in subclause 11.1.] 
11.3.1.2 Chroma transf rm coefficient decoding 

Jomponems" 8 * C ° mp ° nentS f ° r SP and SI non - intra "lacroblocks is similar to the decoding of luma 
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The predicted block, P(x,y), where P(x,y) ={poo ... P33}. is transformed according to Equation 11-1 to produce transform 
coefficients t™. An additional 2x2 transform is applied to the DC coefficients of these blocks as in Equation 11-5: 

The DC transform coefficients are then quantised with a quantisation parameter QS, as in Equation 1 1-10. 

w wed = { Sign(c ij PRED )*[Abs(q j PRED )*Q ij (QS%6) + (l«(16+QS/6))]}»(17+QS/6) 



i,j=0,...,3 



(11-10) 



NOTE - Equation 1 1-10 is the same as Equation 1 1-7 except for the change of name of input and output variables. 

The received prediction residual DC coefficients, w 23 ", are added to these quantised DC transform coefficients of the 

prediction block, as in Equation 1 1-9. 

AC coefficients are decoded in an identical process to that used for luma. 

The value of QP to be used for chroma data, denoted QP C , is obtained from QP r using the relationship specified in Table 
9-1. Similarly, the value of QS to be used for chroma data, denoted QS C , is obtained from QS Y using the relationship 
specified in Table 9-1 

12 Adaptive block size transforms 
12.1 Introduction 

In this clause, the modifications to the syntax and semantics in clause 7 and the changes to the decoding process in 
clause 8 and to entropy coding in clause 9 for adaptive block size transforms are described. 

If adaptive blockJsize_transform_flag = = li additional transforms of size 4x8, 8x4, and 8x8 are specified for the luma 
residual The chroma residual decoding process remains unchanged. Adaptive block size transforms are used for all 
macroblocks with QP y >= 12 In inter predicted macroblocks, the transform block size is indicated by the block size 
used for inter prediction. For intra macroblocks, the block size used for intra prediction is connected to the block size of 
the transformation. For intra macroblocks in inter slices, the block size is indicated by the syntax element 
intra_block_typeABT. For intra slices, the intra block size is indicated by the macroblock block mode. 
Figure 12-1 shows the order of the assignments of syntax elements for luma resulting from coding a macroblock to sub- 
blocks of the macroblock if the ABT features is used. The assignment of blocks and coded J>lock_patternY is specified 
in Figure 12-1. An 8x8 block may contain 1, 2, or 4 transform blocks. An indication that an 8x8 block contains 
coefficients means that the 8x8 transform blocks or one or more of the 2, or 4 transform blocks within the 8x8 block 
contains coefficients. The chroma 4x4 residual blocks are ordered after the luma blocks as indicated in Figure 6-6. 



CBPY 8x8 block order 



0 


i 


2 


3 



Luma residua! coding ABT block order 
for one CBPY 8x8 block 



8x8 



8x4 



4x8 



4x4 



0 


1 


2 


3 



Figure 12-1 - Ordering of blocks for CBPY and luma residual coding of ABT blocks 
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12.2 ABT Syntax 

12.2.1 Macroblock layer syntax 



macroblock_layer_abt( ) { 


Category 


Descriptor 


mb_type 


4 


ue(v) | ae(v)~~ 


if( num_mb_partition[ mb_type ] = = 4 ) 






sub_mb_pred_abt( mb_type ) 


4 




else 






mb_pred_abt( mb_type ) 


4 




SendResidual = 0 






if( mb_partition__pred_mode( , 1) == Intra && 

mb_type != Intra 4x4) [ /* Intra 16x16 X Y Zmb type*/ 






coded_b!ock_pattern 


4 


me(v) j ae(v) 


if( coded_block_pattern > 0 ) 






SendResidual = 1 






} else { 






coded_block_pattern 


4 


me(v) | ae(v) 


if( coded_block_pattern > 0 ) 






SendResidual = I 






) 






if( SendResidual ) { 






if( !mb_frarne_f]eld_adaptive_flag 1 1 
(nib_frame_fleld_adaptive flag && 
( pic_structure = = 0 1 1 

pic_structure = = 3 || pic__structure = = 4 ) && 
first_non_skip mb in pair( ) ) 






delta_qp 


4 


se(v) | ae(v) 


residual_abt( ) 


5 1 6 




} 






J 
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12.2.1.1 Macroblock prediction syntax 



mb_pred_abt( mb_type ) { 


Category 


Descriptor 


if( mb_partition_pred_mode( , 1) = = Intra ) { 






if( mb_type == Intra_4x4 

« a TknnT m i ill 

mb_type == ABTIntra_4x4 || 

mh tvrv* ARTTntra 4x8 1 i 

1 1 iu ijr Lie — — ad x iuu b_*tao | | 

mb_type == ABTIntra_8x4 || 
mb type = = ABTIntra_8x8 ) { 






if( MbABTFlag && (slice_type( ) = = Pred 1 1 slice_type( ) = = BiPred ) 






intra_block_typeABT 


4 


me(v) | ae(v) 


for( i = 0; i < num_mb_intra_partition( ); i++ ) 1* for each luma block */ 






intra_pred_mode 


4 


ce(v) | ae(v) 


} 






intra_chroma_pred_mode 


4 


ue(v) | ae(v) 


} else if( mb_type != Direct_16xl6 ) { 






for( i = 0; i < num_mb_partition( mb_type, i ); i++ ) 






if( num_ref_idx_10_active_minusl >0 && 

mh n2»rtitir»n nrpH mnHpY mh tvne t ^ 1= Pred LI ^ 






refJdxJO 


4 


ue(v) | ae(v) 


for( i = 0; i < n um_mb_parti ti on( mb_type, i ); i++ ) { 






if( num_ref_idx_ll_active_minusl > 0 && 

mb parti tion_pred_mode( mbjtype, i ) !- Pred_L0 ) 






refjdxjl 


4 


ue(v) | ae(v) 


for( i = 0; i < num_mb_partition( mb_type ); i++ ) { 






if( mb_partition_pred_mode ( mb_type, i ) != Pred_Ll ) 






for(j = 0;j<2;j++) 






mvdJOt i][j] 


4 


se(v) | ae(v) 


for( i = 0; i < num_mb_partition( mb_type ); i++ ) { 






if( mb_partition_pred_rnode( mb_type, i ) != Pred_L0 ) 






for(j =0;j <2;j++) 






mvdjl[i][j] 


4 


se(v) | ae(v) 


} 






} 
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12.2.1.2 Sub macroblock prediction syntax 



sub_mb pred abt( mb type ) { 


Category 


Descriptor I 


fblY i — 0* i <T 4* t4-4- ^ /* fnr f*ar\\ cnh morrrxkl/^^L- */ 
* — 1 ^- ff i~tt ) 1 iui cucii aiiu macrooiOCK / 






sub_mb_type[ i ] 


4 


ue(v)|ae(v) 


IntraChromaPredModeFlag = 0 




j 


fbrf 1 — 0" 1 < 4* 14-+ ^ /* finr pnrh Qiih marrnK1/vl- */ 
1V, "\ * » ^ * » » * » ,/ / lul Csdl/ll oUU iJidCiUDlOCK. / 




| 


iff sub mb tvnef i 1 Intra Ry8 ^ / -— — — 

1 1 V * s "*-'__ ail *-'_ | 'jr p'fc'l 1 J — — .Mill a. O AO j 1 






iff MbABTFlaz }f 




— i 

i 


intra block tvnpART ~ 


4 


me(v)|ae(v) 


IOr( \ — Ot I < niim ^iiH mh infra r\c\rttt'tr\r\f V i . i \ f 




1 


intra nrpH mnrlp 


4 


ce(v) | ae(v) j 


IntraChromaPredModeFlag = 1 












} 






if( IntraChromaPredModeFlag ) 






intra__chroma pred mode 


4 


ue(v)|ae(v) 


ror( 1 = u; 1 < 4; 1++ ) /* for each sub macroblock */ 




[ 


if( num_ref_idx_10_active_minusl >0 && 
mb_type != Pred_8x8ref0 && 
sub_mb_type r i ] != Intra_8x8 && 
sub_mb_type[ i ] != Direct_8x8 && 
sub_mb_pred mode( sub mb typer i 1) != Pred LI > 




J 


ref_idx__10 


4 


ue(v)|aefv) 


for( i = 0; i < 4; i++ ) /* for each sub macroblock */ 1 




j 


if( num_ref_idx_ll_active_minusl > 0 && 
( sub_mb_type[ i ] != Intra_8x8 && 
sub__mb_type[ i ] != Direct_8x8 && 
sub_mb_pred_mode( sub mb type[ i ] ) != Pred L0 ) 




1 


rei_iax_JI 1 


4 


ue(v) | ae(v) 


rort 1 — u; 1 < 4; 1++ ) /* tor each sub macroblock */ 




] 


u\ suD_mo_type[ 1 J !— Intra_8x8 && 
sub_mb_type[ i ] != Direct_8x8 && 
sub_mb_pred mode( sub mb typef i ] ) != Pred LI ) 






for( j = 0; j < num_sub mb partition( sub mb typer i 1 ); i++ ) 




j 






! 


mva_iUL * Jl J Jl k J 


4 


se(v) | ae(v) 


,UI v i - i *++ j /*•* ror eacn suo macroblock */ 






if( sub_mb_type[ 1 ] != Intra_8x8 && 
sub_mb_type[ i ] != Direct_8x8 && 
sub_mb_pred_mode( sub mb typef i ] ) != Pred LO ) 






for(j =0; j <num sub mb partition( sub mb tYDer i H: i++ ^ 






for(k = 0;k<2;k++) K ^ 






mvdjl[i]fjirkl ' 


4 


se(v) | ae(v) f 


) 
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12.2.1.3 Residual data syntax 



residual abt( mb type ) { 




Descriptor 


if( entropy coding_mode = = 1 ) { _ j 






residual__4x4block = residual_4x4block_cabac( ) / 5,! tunction pointer / 


5 16 




residual subblock = residual_subblock_cabac( ) tuncuon pointer / 


5 16 




} else { 






residual_4x4 block = residual_4x4block_cavlc( ) tunction pointer / 


5 16 




residual_subblock = residual_subbIock_cavlc( ) /* tunction pointer / 


5 16 




} 






if( mb type == Intra_16xl6) 






residual 4x4block( intra 16x1 6DC, 16) 


5 




for( 18x8 = 0; 18x8 < 4; 18x8++ ) /* each luma 0x5 oiock / 






for( i4x4 = 0; i4x4 < num_sub_blocks( ); i4x4++ ) / eacn sud-diock or oiock / j 






if( coded__block_pattcrn & ( 1 « i8x8 ) ) 






if( mbjype = = Intra_16xl6 ) 






residual_4x4block( intra 16xl6AC, 16 ) 






else 






if( MbABTFlag && ( siice_type() = = Intra 1 1 mb„type = = lntra_«x4 ; 






residual_subblock( IntraABT, sub_block_type ) 


D j O 





" Ti " v 

else 






residual_subblock( Inter ABT, sub_block_type ) 


S 1 €% 

D | D 




if( coded_block_pattern & 0x30 ) /* chroma DC residual coded */ 






for( iCbCr = 0; iCbCr < 2; iCbCr++ ) 






residual_4x4block( chromaDC, 4 ) ) 






iff t+t\A/*t\ HlnrW r»artprn &. 0x20 ^ t* chroma AC residual coded */ 






for( iCbCr = 0; iCbCr < 2; iCbCr++ ) 






For( i4x4 = 0; i4x4 < 4; i4x4++ ) 






residual_4x4block( chromaAC, 16) 


5|6 




} 
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residuaLsubb!ock_cavlc( b!ock_coding_type, sub_bIock_type ) { 






if( ! MbABTFlag ) 






residuaL4x4block_cavlc(Iuma, 16) 


5 J 6 




else { 






if(block_coding_type = = IntraABT ) { 






num_coeff_abt 


5 |6 


ce(v) 


for ( i=0; i<num_coeff_abt; i++ ) { 






code__nu ruber 


5 | 6 


ce(v) 


if( code_n umber != escape ) { 






levelf i J = code_number2IeveUntra( code_number, sub_block type ) 






run[ i ] = code_number2run_intra( code_number, sub_block type ) 







} else { 







code_number 


5|6 


ce(V) 1 


level[ i ] = escapejevelf code_number ] 






codejsumber 


5|6 


cefv} 1 

V v / I 


run[ i ] = escape_run[ code_number ] 






} 







} 







} else { 







for( i=0; i<max_numcoeffABT(subJ>lock_type); i++ ) { 







code_n umber 


5 | 6 


ce( v^ 1 


if( code_number ==0) 






break; 







if( code_n umber != escape ) { 







run[ i ] = code_nurnber2run_inter( code_number, sub_block_type ) 







levelf i ] = code_number21eveLinter( code_number t sub_block type ) 







} else { 







code_number 


5|6 


ce(v)j 


Ievel[ i ) = escapejevelf code_number ] 






code_number 


5|6 


ce(v) 1 


runf i ] = escape_run[ code_number ] ~f 






} 






) 






} 






} 
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12.2.1.3.2 Residual sub block CABAC syntax 



residuai_subDlock_caoacQ Diocic__coaing_iype, suo_DiocK_iype j i 


r^afpo rv 




if( ! MbABTFlag ) 






resiaual_4x4block_caoac(. luma, 10 ; 






else { 


— : 




if( sub_block_type != oxe ) 






cbp4 


5 16 




if( cbp4 1 1 suo_oiocK_type — oxo ) \ 






max__numcoeff — max_num_coeff„abt( sub_block__type ) 






significant_coeff[ max_numcoeff - 1 ] = 1 






for( i = 0; i < max_numcoeff; i++ ) { 






signiiicant_coeiiL i J 


5 16 


ttC v v / 


if( signiiicant_coeiit 1 J ococ 1 < rnax_numcoerr- 1 ^ i 






Iast_signi£icant__coeffI i ] 


<5 1 6 




if( last_significant_coeff[ i ] ) 






max_numcoeff = i + 1 






} 






) 






for( i - max_numcoeff-l; i >= 0; i— ) 






, if( sigmficant_coeff[ i ] ),{ 






coeff_absolute_value_minusl[ i ] 


<N 1 fx 


ae(v) 


coeii_sig*H t j 


5 | 6 


aefv^ 
uv,, \ t / 


} 






} 






} 






} 







123 ABT Semantics 

12.3.1 Macroblock layer semantics 

MbABTFlag indicates the usage of ABT for the macroblock. If adaptive jDiock_transform_flag is equal 1 and QP Y >= 
12* MbABTFlag = 1 else MbABTFlag = 0. 

The meaning of mb_type Tntra_4x4* in Inter slices is modified for ABT. If MbABTFlag is equal 1 , this macroblock type 
indicates application of ABT Intra prediction and transformation. In Inter slices, the block size used for prediction and 
transform is indicated by the syntax element intra_block_typeABT. For I slices, the prediction and transform block size 
is derived from the macroblock mode. The modified macroblock types for I slices are specified in Table 12-1 below. 



Table 12-1 - Mollified macroblock types for I slices 



Value of 
mb_type 


Name of mb_type 
in I slices 
if MbABTFlag = = 1 


mb_partition_pred_ 
mode( , 1) 


0 


ABTIntra_4x4 


Intra 


1 


ABTlntra_4x8 


Intra 


2 


ABTIntra_8x4 


Intra 


3 


ABTIntra_8x8 


Intra 
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12.3.1.1 Macroblock predicti n semantics 

num_mb_intra_partition( ) depends on mb_type in I slices and on intra_block_typeABT in P slices. The assignment of 
num_mb_jntra_partition( ) is specified in Table 12-2. 

intra_block_typeABT indicates the blocksize used for ABT intra decoding in Inter slices. Blocks of size 4x4, 4x8, 8x4 
and 8x8 samples may be used for ABT intra prediction. Table 12-2 provides num_mb_intra_partition' and 
num_sub_mb_intra_partition specifying the number of intra_pred„mode syntax elements to be decoded. 



Table 12-2 - ABT intra partitions 



mb_type 


intra_block_typeABT 


num_mb_intra._partition( ) 


num_sub_mb_intra_partition( ) 


ABTIntra_4x4 


4x4 


16 


4 


ABTIntra_4x8 


4x8 


8 


2 


ABTlntra_8x4 


8x4 


8 


2 


ABTIntra_8x8 


8x8 


4 


1 



12.3.1.2 



Sub macroblock prediction semantics 



If MbABTFlag is equal 1, the number of decoded intra prediction modes is indicated by num_sub_mb_intra partition( ) 
as specified in Table 12-2. 



12.3.1.3 



Residual data semantics 



num_sub_blocks( ) indicates the number of partitions in a luma 8x8 block. If MbABTFlag is equal 0 num_sub_blocks is 
4. If MbABTFlag equals 1, num_sub_blocks may be 1, 2, or 4 depending on sub_block_type as specified in Table 12-3. 

Table 12-3 - ABT Intra Block Types 



sub__block_type 


num_sub_b!ocks( ) 


max_num_coeff_abt( ) 


4x4 


4 


16 


4x8 


2 


32 


8x4 


2 


32 


8x8 


1 


64 



sub_block_type indicates the block type used for decoding of an 8x8 block. The decoded blocks may be of size 4x4, 
4x8, 8x4, or 8x8 samples. For intra slices with MbABTFlag ==1, sub_block_type for macroblock type 
ABTIntra_NxM is NxM. For intra macroblocks and intra sub macroblocks in inter slices sub_block_type is equal to 
intra_block_typeABT. For inter macroblocks with macroblock type Pred_x_NxM or Pred_x_y_NxM or BiPred_x_NxM 
or BiPred_x_y_NxM, x, y = L0, LI, Bi, sub_block_type is N\M'. N* and M' are derived from N and M by clipping: 

N'=Ch P 3(4,8,N) (12-1) 



M*=Clip3( 4,8,M) 



For sub macroblocks with sub macroblock type Pred_x_NxM or Pred_x_y_NxM or BiPred_xJMxM or 
BiPred_x_y_NxM, x, y = L0, LI, Bi, sub_block_type is NxM. 

For macroblocks and sub macroblocks that are predicted in direct mode, sub_block_type is derived from the co-located 
macroblock or sub macroblock. If MbABTFlag is equal 1 for the co-located macroblock, sub_block_type for the current 
macroblock is derived from the co-located macroblock type or sub macroblock type as specified above. If MbABTFlag 
is equal 0 for the co-located macroblock and the type of the co-located macroblock is not Intra_16xl6, sub_b!ock_type is 
equal 4x4. If MbABTFlag is equal 0 for the co-located macroblock and the type of the co-located macroblock is 
Intra_16xI6, sub_block_type is equal 8x8. If the co-located macroblock type is MbSkip, sub_block_type is equal 8x8. 
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block_coding_type indicates if an Inter or an Intra block is decoded. Inter blocks are indicated by block_coding_type = 
TnterABT, intra blocks are indicated by block_coding_type = IntraABT. 

123.13.1 Residual sub block CAVLC semantics 

num_coelT_abt indicates the number of coefficients to be decoded. num_coeff_abt is bound by max_num_coeff_abt( ) " 
specified in Table 12-3. 

cod _number indicates the number of the decoded codeword. The code structure of the codewords depending on the 
syntax element to be decoded is specified in subclause 12.4.1 . 

code_number2run_intra( ) retrieves run from Table 12-9 for Intra blocks dependent on QP Y . 
code_number2run_inter( ) retrieves run from Table 12-9 for Inter blocks. 
code_number21eveUntra( ) retrieves level from Table 12-9 for Intra blocks dependent on QP Y . 
code_number21eveLinter( ) retrieves level from Table 12-9 for Inter blocks. 

escape indicates separate( ) decoding of level and run. The value of escape is specified in subclause 12.5.1.2.2. 
escape Jevel( ) retrieve level symbol after escape as specified in Table 1 2-8. 
escape__run( ) retrieve run symbol after escape as specified in Table 1 2-6. 
12.3.13.2 Residual sub block CABAC semantics 

The syntax elements of residual_subblock_cabac( ) are specified in subclause 7.4.5.3.2. 
12.4 ABT decoding process 

1 v 

12.4.1 Intra Prediction for 4x8, 8x4, and 8x8 luma blocks 
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Figure 12-2 - Identification of samples used for ABT intra spatial prediction for 4x8, 8x4, and 8x8 luma blocks 



Figure 12-2 illustrates the intra prediction for 4x8, 8x4, and 8x8 blocks that may be used in addition to the 4x4 intra 
prediction specified in subclause 8.5. The samples p™, with m=0 to M-l and n=0 to N-l, M,N = {4,8}, are predicted 
using samples t k> k=0 to (N+M-l), q, and l k , k=0 to (N+M-l), from neighbouring blocks. 

Samples t k , k=0 to (N+M-l) or samples l k , k=0 to (N+M-l) shall be considered not available under the following 
circumstances: 

- if they are outside the picture or outside the current slice, 

if they belong to a macroblock that is subsequent to the current macroblock in raster scan order, 
if they are sent later than the current block in the order shown in Figure 12-1, or 
if they are in a non-intra macroblock and constrained_jntra_pred is 1. 

When samples t k , k=N to (N+M-l) are not available the sample value of t^i is substituted for the samples t k , k=N to 
(N+M-l). When samples l k , k=M to (N+M-J) are not available the sample value of l M .i is substituted for the samples l kt 
k=N to (N+M-l). 



12.4.1.1 



Mode 0: vertical prediction 



block size 




samples 


predicted by 


4x8 


k = 0 to 7, i = 0 to 3 


Pw 


(t M + tj«l +2)»2 


8x4 


k = 0 to 3, i = 0 to 7 


8x8 


k = 0to 7,i=0to7 



12.4.1.2 



Mode 1: horizontal prediction 



block size 




samples 


predicted by 


4x8 


k = 0 to 7, i = 0 to 3 


Pki 


( 1 M + lj«l + l.+l + 2 ) » 2 
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8x4 


k = 0 to 3, i = 0 to 7 






8x8 


k = 0 to 7, t = 0 to 7 



12.4.13 M de 2: DC prediction 

For DC prediction, all samples p w are predicted by the same value p. 



«-» J CUV « T 

block size 


P 


4x8 


( ( to + t, + t 2 + t 3 + 2 ) » 2 + ( 1 0 + h + 1 2 + 1 3 + U + Is + h + 1? + 4 ) » 3 ) » 1 


8x4 


( ( to + 1, + t 2 + h + U + t 5 + t6 + t 7 + 4 ) » 3 + ( I 0 + li + 1 2 + I3 + 2 ) » 2 ) » 1 


8x8 


(to + 1, + 1 2 + 1 3 + U + t 5 + U + t 7 + 1 0 + li + h + h + >4 + U + 16 + 1? + 8 ) » 4 



b) If l k , k = 0 to (M-l) are availabl e and t k , k == 0 to (N-l) are not available 



block size 


P 


4x8 


( 1 0 + li + I2 + h + U + Is + 1* + h + 4 ) » 3 


8x4 


(l 0 + i| + l 2 + l 3 + 2)»2 


8x8 


do + li + 12 + I3 + U + »s + le + 1? + 4 ) » 3 



" VBW TP 

block size 


P 


4x8 


( I© + t, + t 2 + 1 3 + 2 ) » 2 


8x4 


( to + 1, + 1 2 + t 3 + t4 + t 5 + tt + t 7 + 4 ) » 3 


8x8 


(to + ti + 1 2 + 1 3 + U + t 5 + t6 + t 7 + 4 ) » 3 



d) If l k , k = 0 to (M- 1 ) are not available and t k , k = 0 to (N-l) are not available 



block size 


P 


4x8 


128 


8x4 


128 


8x8 


128 



12.4.1.4 Mode 3: diagonal down/left prediction 



4x8 block samples 


8x4 block samples 


" v- - 1 _ 

8x8 block samples 


predicted by 


Poo 


Poo 


Poo 


( ( b + h«l + lo + 2)»2 + (lo + + 1 2 + 2 ) » 2 ) » 1 


pot. pio 


P01. pio 


poi. pio 


( ( b + h«l + li + 2 ) » 2 + ( t, + t 2 «l + 1 3 + 2 ) » 2 ) » 1 


P02, Ptl. P20 


p02. Pll. P» 


p02. pll. P» 


( ( U + l3«l + h + 2 ) » 2 + ( t 2 + ta«l + U + 2 ) » 2 ) » 1 


P03, p!2» p21t P» 


pO*. P>2» P2U P30 


PQOh pll. P2I. P» 


( ( b + U«l + 1 3 + 2 ) » 2 + ( tj + LK<1 + 1 5 + 2 ) » 2 ) » 1 


Ptt, P22. f>3l. P«0 


P04, Pl3. P22. p31 


P04, pi), PZ2> P31. P«0 


( ( U + \s«l + U + 2)»2 + ( t* + t $ «l + Us + 2)»2)»I 


P23. P32, P41. P» 


P05» P»4» p23. Pl2 


pas pu, Pzy. P32. P*i» P» 


( ( h + U«l + Is + 2 ) » 2 + ( t 5 + t6«l + t 7 + 2 ) » 2 ) » 1 


P33, P<2, pSIt PfiO 


p06r Pl5. pW. pJ3 


P06.Pl5.P34.P35. P41. P5I. P«0 


( ( Is + 1t«1 + U + 2)»2 + (t6 + t7«l + t« + 2 ) » 2 ) » 1 
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P<3» P52» P6I» P70 


p07t Pl6» p25, p34 


P07, p!6, P25, PX, p43, p52» P61, P70 


( ( I 9 + J««l + 1 7 + 2 ) » 2 + ( t 7 + U«l + t 9 + 2 ) » 2 ) » 1 


PS3» p62? P7I 


Pl7« P26» Pas 


Pl7. p26, p35. . p44, p53» p62, p71 


( ( I10 + l9«l + 1« + 2 ) » 2 + (U + t9«l + t J0 + 2 ) » 2 ) » 1 


p63* p72 


p27» p36 


P27, P36> P*5» P54» P«> PT2 


( ( In + ho«l + I9 + 2 ) » 2 + ( t 9 + tio«l + tn + 2 ) » 2 ) » 1 


On 




P37» P46» p55» P64» p73 


( ( Ip + + 110 + 2 ) » 2 + ( tto + tn«l + ti2 + 2 ) » 2 ) » I 






P47, P56» P65» P74 


( ( il3 + Il2«l + 111 + 2 ) » 2 + ( In + tp«l + + 2 ) » 2 ) » 1 






P57, p66> P7S 


( ( 1)4 + + In + 2 ) » 2 + ( tn + tn«l + + 2 l » 2 1 >:> 1 






p67, p76 


( ( 1|3 + ll4«l + 1.3 + 2 ) » 2 + ( 1,3 + t, 4 «l + ti 5 + 2 ) » 2 ) » 1 






P" 


((li5 + lis«l +1m + 2)»2 + (t, 4 + t, 5 «l + t,s + 2)»2)» 1 



12.4.1.5 



Mode 4: diagonal down/right prediction 



4Xo DiocK samples 


8x4 block samples 


8x8 block samples 


predicted by 




Po? 


P07 


(t 3 + t 6 «l + t 7 + 2)»2 




P06. p|? 


P06,pl7 


( t4 + t 5 «l + t6 + 2)»2 




P03» KI6» P27 


P05» p!6» P27 


\ ly + U« I + t$ + 2 ) » 2 




P04mPI5» p26, P37 


P04, Pl5, P26, P37 


( t 2 + t 3 «l + 14 + 2)» 2 


P03 


P03. Pl4» p25, P36 


P03, P14, p25, P36, P47 


( ti + t 2 «l + t 3 + 2 ) » 2 


P02. Pl3 


P02, PWt P24. p35 


P02. Pl3, P24. P35, P46, p57 


( to+ti«l + t 2 + 2)»2 


P01» p!2, P23 


POI, P«, p23, P34 


P01, Pl2, p23. P34. , p45, PS6, P«7 


(q + lo«l +ti + 2)»2 


Poo, pin P22. P33 


POO, pll. p22, P33 


POO, Pll. D22> P33. P44. p55, p66> P77 


(lo + q«l +t 0 + 2)»2 


PlO» P2I» P32. p43 


PlO, p21. p32 


PlO, P21, P32. . P43, P«, P65, p76 


( it + lo«l + q + 2 ) » 2 


P20, P31» P42> P33 


P20, P31 


P20, p3l, P«, P33, P64, P75 


( h + li«l + l 0 + 2 ) » 2 


P30, P41, P52, p63 


P30 


P30, p4l, p52, P63. D74 


( 1 3 + l 2 «l + 1, + 2 ) » 2 


P40» P5I, P62, p73 




P40, P5U P62, p73 


(U + l 3 «l +l 2 + 2)»2 


P50. p6l. P72 




pJOt P6I, p72 


( l 5 + l4«l + I 3 + 2 ) » 2 


P60» p71 




P60, P71 


(U + l 5 «l +l, + 2)»2 


Pro 




P70 


(1 7 + U«1 +I 3 + 2)»2 



12.4.1.6 



Mode 5: vertical-left prediction 



4x8 block samples 


8x4 block samples 


8x8 block samples 


predicted by 




P07 


P07 


((t6 + t7«l +t 8 + 2)»2 + (t 7 + t,«l +t 9 + 2)»2)» 1 




P" 


Pl7 


(t6 + t7«l + U + 2)»2 




P06>P27 


P06, P27 


((t s + t6«l +t 7 + 2)»2 + (t6 + l 7 «l +u + 2)»2)» I 




Pl6, P37 


Pl6, P37 


(t5 + t<s«l +t 7 + 2)»2 




P03, P26 


P05. P26, p47 


( ( U + t 5 «l + t6^2)»2 + (t 5 + t6«l + 1 7 + 2 )» 2 )» 1 




Pl5. P36 


Pl5, P36, P57 


( U + t 5 «l + t6 + 2 ) » 2 




P04, P25 


P04, P25, P46, P67 


( ( t3 + U«l +t 5 + 2)»2 + (14 + t 5 «l + 1 6 + 2 ) » 2 ) » 1 
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- 


p14* Pl5 


pl4. p3S. P56. P77 


( t 3 + U«l + tj + 2 ) » 2 


Pro 


p03. p24 


D03, p74, R45. p66 


(<t 2 + t 3 «I + U + 2)»2 + (t 3 + L,«l +l 3 + 2)»2)» I 


Pn 


Pl3. P34 


Pl3» P34, p5S. P76 


( h + tj«l + U + 2 ) » 2 


P02. Pn 


p02. PB 


P02, p23* P44, p6S 


((t, + t2«! + t 3 + 2)»2 + (t 2 + t 3 «l + t» + 2)»2)» 1 


Pl2>P33 


pn* P33 


Pl2* p33* p54, p75 


( ti + t2«l + I3 + 2 ) » 2 


Poi. pn. P«3 


poi, pn 


Poi, Pn, p43* p64 


( ( to + ti«l + t 2 + 2 ) » 2 + ( t, + t 2 «l + h + 2 ) » 2 ) » 1 


pll, PJ2, P53 


Pll* P32 


Pll. P32. P53. p74 


( to + ti«l + t 2 + 2 ) » 2 j 


POO. Pll. p42, P63 


poo. P21 


POO. Rai* P43, p63 


((lo + q«l +to + 2)»2 + (to + t,«l + t 2 + 2)»2)» 1 


PlO, Pll, PS2* P73 


pio. pii 


PlO, P31. p52, P73 


( lo + q«i + to + 2 ) » 2 


P20.P41. pe 


P20 


P20. p41. p67 


( 1 2 + + lo + 2 ) » 2 


P30.PS1. P72 


P30 


P30> P51- P72 


< lj + l 2 «! + 1, + 2 ) » 2 


p40» P61 


- 


p40» P61 


( U + l 3 «l + 1 2 + 2 ) » 2 


P50. P71 


- 


P». P71 


(l s + U«t +l 3 + 2)»2 


P60 




P» 


(U + l5«I +L, + 2)»2 


P70 




P70 


( 1 7 + U«I + 1 5 + 2 ) » 2 



12.4.1.7 



Mode 6: horizontal-down prediction 



4x8 block samples 


8x4, block samples 


8x8 block samples 


predicted by 




P07 


P07 


( t 5 + t««l + t 7 + 2 ) » 2 




Pos 


P06 


( U + t 5 «l + t6 + 2 ) » 2 




P03.pl7 


P05* Pl7 


( t 3 + U«l + t 5 + 2 ) » 2 




P04. Pl6 


P04. pl6 


( *2 + ty«l -4- u + 2 ) » 2 


P03 


P03» PI5. P27 


P03, Pl3» P27 


( U + + h + 2 ) » 2 


P02 


P©2. Pl4. D36 


P02. Pl4, DM 


( to + ti«l + 1 2 + 2)» 2 


Pot. Pl3 


P01. Pu* Pn. P37 


pOI, Pl3. P2S. P37 


( q + to«l + ti + 2 ) » 2 


poo. P12 


POO. Pll, P24, P36 


POO, Pl2. P24. p36 


( ( 1 0 + q«l + to + 2)»2 + ( q + lo«l + li + 2 ) » 2 ) » 1 


Pio. Pn 


pio. Pn, P34 


PlO. p22.,p34, D46 


( ( q + lo«l +l| + 2)»2 + (lo + li«l + 1 2 + 2 ) » 2 ) » 1 


Pn. Pn 


pu* Pn* P35 


Pu. Pn. p35, D47 


( q + lo«l + li + 2 ) » 2 


P20. Pn 


P20, pn 


020. P32. P44, P56 


((lo + li«l + l 2 + 2)»2 + (1, +l 3 + 2)»2)» 1 


P21. P33 


pji.pn 


P21. pn. P43* P$7 


( lo + li«l + h + 2 ) » 2 


p30, P<2 


P» 


P30.p42.p54.p66 


((li + b«l +l 3 + 2)»2 + (l 2 + l 3 «l+l4 + 2)»2)»l 


P3I. P43 


P31 


P31. p43, PS3. P67 


( 1, + l 2 «l + 1 3 + 2 ) » 2 


p40.pS2 




P40, P32. P64. p76 


((h + l 3 «l + U + 2)»2 + (l 3 + l4«l + I3 + 2 ) » 2 ) » I 


P41. PS3 




p4l, p53. P65. P77 


( 1 2 + lr«l + U + 2 ) » 2 


Pso-pc 




P90.P62.P74 


((l 3 + l4«l +l 3 + 2)»2 + (l4 + l 5 «l + U + 2)»2)» 1 


P51. p63 




P5I» P63, P» 


(1 3 +U«1 +l 5 + 2)»2 
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P«), P72 




P60. p72 


( ( 14 + l5«l + U + 2 ) » 2 + ( 13 + l6«I + 17 + 2 ) » 2 ) » 1 


P61» P73 




P61» P73 




P70 




Pro 


( ( l 5 + + 1 7 + 2 ) » 2 + + 1t«1 + h + 2 ) » 2 ) » 1 


P7I 




P7I 


(l 5 + l 6 «l + l 7 + 2)»2 



12.4.1.8 Mode 7: vertical-right prediction 

t k , k = 0 to (N-l) shall be available. 



HaO UIvJCK. i><lillJJICc> 




OAO UiUCK J>dJlipidI» 


prcuicicu oy 


Poo 


poo 


Poo 


{, I to + + 12 + 2 ) » Z + ( t| + t 2 «I + t 3 + 2 ) » 2 ) » 1 


pio 


Pio 


pio 


( K\ + t?«i + t 3 + 2 ) » 2 


poi. P20 


poi. P20 


poi, P20 


( ( ti + t 2 «l + t 3 + 2 ) » 2 + ( t 2 + t 3 «l + t4 + 2)»2)» I 


pi 1. P30 


Pll. P30 


pn, P30 


( t 2 + tj«\ + U + 2 ) » 2 


P02, P21. P40 


P02, P21 


P02, p2l, p40 


( ( t 2 + t 3 «l + U + 2)»2 + (tj + U«i + 15 + 2 ) » 2 ) » 1 


Pl2» P3I, P50 


Pl2, P3I 


Pl2, P3I. P50 


( t 3 + U«l + t 5 + 2 ) » 2 


P03, f>22, P4I. P60 


P03. P22 


P03. p22. p41, p60 


( ( t 3 + U«l + |s + 2 ) » 2 + ( u + t 5 «l + t6 + 2)»2)»1 


Pl3» P32, P51, P>70 


Pl3» P32 


Pl3, P32, P51. p70 


( t, + ts«I + U + 2 ) » 2 


P23, p42, P61 


P04, p23 


P04, P23, P42, P61 


( ( to + ti«l + 1 2 + 2 ) » 2 + ( ti + t2«l + t 3 + 2 ) » 2 ) » I 


P33» P52, p71 


Pl4, p33 


Pl4, p33» P52. P71 


( t 5 + t*«l + t 7 + 2 ) » 2 


p43, P62 


p05, P24 


P05t P24. p43, P62 


( ( t 5 + tg«l +t 7 + 2)»2 + (t6 + t 7 «l + ts + 2 ) » 2 ) » 1 


P53, P72 


Pl5» P34 


Pl5, P34, p53, p72 


( 16 + 17« 1 + h + 2 ) » 2 


P63 


P06» P25 


P06, P25» p44. p63 


i V 16 + 17«I + 18 + ^ ) » + ( 17 + + 19 + JL ) » Z » 1 






Dix, D"*<, Dv*, D71 


( t7 + tg« ] + t9 + 2 ) » 2 




p07. P26 


P07» P26» P45, P64 


( ( t 7 + t,«1 + 1 9 + 2 ) » 2 + ( t« + to«l + ho + 2 ) » 2 ) » 1 




p!7. P36 


Pl7, p36. P55» P74 


( + ty«l + t, 0 + 2 ) » 2 




P27 


P27, p46. p6S 


( ( ts + t 9 «1 + 1, 0 + 2 ) » 2 + ( t 9 + t t0 «l + tu + 2 ) » 2 ) » 1 




P37 


p37» P56. p75 


( to + ti 0 «l + t M + 2 ) » 2 






P47, p66 


( ( t 9 + t l0 «l + t„ + 2 ) » 2 + ( t, 0 + t„«] + 1, 2 + 2 ) » 2 ) » 1 






P57. P76 


(tio + tn«l +t| 2 + 2)»2 






P67 


((tio+tn«l +ti2 + 2)»2 + (tu + ti2«l +t| 3 + 2 )»2)» 1 






P77 I 


(t n + t| 2 «l +t l3 + 2)»2 



12.4.1.9 Mode 8: horizontal-up prediction 

l k , k = 0 to (M-l) shall be available. 



4x8 block samples 


8x4 block samples 


8x8 block samples 


predicted by 


Poo 


poo 


Poo 


((lo+l,«l +l 2 + 2)»2 + (1, + l 2 «l +1 3 + 2)»2)»1 


P0! 


Poi 


Poi 


( li + l2«l + 1 3 + 2 ) » 2 


PIO, P02 


PlO, P02 


Pio, P02 


( ( 1, + 1 2 «1 + 1 3 + 2 ) » 2 + ( l 2 + 1 3 «1 + U + 2 ) » 2 ) » 1 


Pll, P03 


Pit. P03 


Pll» P03 


(1 2 + 1 3 «1 +L, + 2)»2 
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P20t PU 


P». Pi2* P04 


pao. pn. poi 


((i2 + l3«l +U + 2)»2 + (1 3 + Li«l +! 5 + 2)»2)» 1 


P2I* Pl3 


p2l, Pl3, POS 


piu P13. pos 


( 1 3 + l4«I + 1 3 + 2 ) » 2 


pJO, PZ2 


P*H P22, pI4, p06 


P30. P22. Pl4« p06 


((l, + l4«! + l 5 + 2)»2 + (L, + I s «l +U + 2)»2)» I 


P31t p23 


P3I, P23, pl5» P07 


P3I. P23* PlJ. P07 


(U+ls«I +U + 2)»2 


p4o» Pn 


P32, P24. pl6 


p40. P32. P24. pl6 


((U + l 5 «l +I« + 2)»2 + (1 5 + U«I +h + 2)»2)» 1 


P41, p33 


P33. P». Pl7 


P4I, P33. P2S» Pl7 


(1 5 + U«I +b + 2)»2 


P50. p43 


P34.P36 


pSO. P42. p34, p26 


((I 5 + l6«l +l 7 + 2)»2 + (U + I 7 «l + I« + 2)»2)» 1 


pSI» P43 


P35. P27 


P51» P43. p35» p27 


(i« + l7«l +l t + 2)»2 


P60.PS2 


P36 


P60. P32. P44, p36 


( ( U + 1t«1 + U + 2 ) » 2 + ( 1 7 + + l 9 + 2 ) » 2 ) » ! 


P6!»PS3 


P37 


P6I. P53* p45» P37 


(l 7 + It«I +U + 2)»2 


P70, p62 


- 


P70. P62, P54, P46 


((l7 + lt«l +l 9 + 2)»2 + (1, + +l,o + 2)»2)» 1 


P7U P63 


- 


P7I. P«» P35» p47 


( Is + k«l + 1 10 + 2 ) » 2 


Pn 




PH, P64, p56 


( ( l a + + l,o + 2 ) » 2 + ( 1 9 + tio«l + In + 2 ) » 2 ) » 1 


P73 


- 


P73, P65, P57 


( 1» + lio«l + In + 2 ) » 2 




- 


P74, p6 


( ( I» + 1 10 «1 + In + 2 ) » 2 + ( lio lu«l + I12 + 2 ) » 2 ) » I 


- 


- 


V 

P75» P67 


( I,o+I n «l +l, 2 + 2)»2 






P76 


( ( ho + ln«l + I12 + 2 ) » 2 + ( 1., + l 12 «l + 1,3 + 2 ) » 2 ) » 1 






P77 


( 111 + Il2«l + ll3 + 2 ) » 2 



12.4.2 Scanning method for ABT blocks 

Scanning patterns for blocks of size 4x4, 4x8, 8x4, and 8x8 coefficients are given below. For blocks decoded in frame 
mode, the zig-zag scans are used. For block decoded in field mode, the field scans are used. The zig-zag scan for 4x4 
blocks corresponds to the zig-zag scan specified in Figure 8-12. 

12.4*2*1 Zig-zag scan 
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Figure 12-3 - 4x4 zig-zag scan 



0 


2 


3 


9 




4 


8 


10 


5 


7 


11 


17 


6 


12 


16 


18 


13 


15 


19 


25 


14 


20 


24 


26 


21 


23 


27 


30 



142 ITU-T Rec. H.264 (2002 E) 

<SOOCID: <XP 1 1 00641 A_l_> 



ISO/IEC FCD 14496- 



22 


28 


29 


31 



Figure 12-4 - 4x8 zig-zag scan 
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Figure 12-5 - 8x4 zig-zag scan 



0 


1 


5 


6 


14 


15 


27 


28 


2 


4 


7 


13 


16 


26 


29 


42 


3 


8 


12 


17 


25 


30 


41 


43 


9 


11 


18 


24 


31 


40 


44 


53 


10 


19 


23 


32 


39 


45 


52 


54 


20 


22 


33 


38 


46 


51 


55 


60 


21 


34 


37 


47 


50 


56 


59 


61 


35 


36 


48 


49 


57 


58 


62 


63 



Figure 12-6 - 8x8 zig-zag scan 



12.4.2.2 Field scan 
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Figure 12-7 - 4x4 field scan 
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Figure 12-8 - 4x8 field scan 
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Figure 12-9 - 8x4 field scan 
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Figure 12-10 - 8x8 field scan 



12.43 Scaling and inverse transform for ABT blocks 

The scaling and inverse transform of residual blocks of block size larger than 4x4 is specified telow. The scaling and 
inverse transform for 4x4 blocks is specified in subclause 8.6. For 8x8 blocks, the coefficients Rf , used in the 
formulas below, are defined as: 



8*8 



(12-3) 



where the subscript of V*** is the row index of the vector defined as: 



ySxt _ 



15 
17 
19 

22 
24 
27 



(12-4) 
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For 4x8 blocks, the coefficients J??. , used in the formulas below, are defined as: 

tfmyJ v tt 4 " for < = 0,...,7; 7=0,2 
# 1^,4.4,8 for / = o,...,7;y = l,3 

where the first and second subscripts of V 8 " 4 4x8 are row and column indices, respectively, of the matrix defined 



(12-5) 



as: 



,8x4,4*8 



"9 


If 


10 


12 


11 


14 


12 


16 


14 


17 
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20 



(12-6) 



For 8x4 blocks, the coefficients R~ m) , used in the formulas below, are defined 

; y 8x4, 
mO 
i7 8x4,' 



as: 



l ' 4 * 8 for , = 0,2;y=0,„.,7 
148 for i = l,3; j=0,...J 



(12-7) 



where the first and second subscripts of V 8 * 4 - 4 ** are row and column indices, respectively, of the matrix defined in 
Equation 12-6. 

The coefficient levels are multiplied with the scaling value R 

^j^[c iJ 'Rjf P%6) ]« (QP/6-2), / = 0,..,AT,y=0,...,M (12 . 8) 

After constructing an entire MxN block of scaled transform coefficients and assembling these into a MxN matrix W of 
elements wg illustrated as 



W = 



'00 



0(/V-|) 



(12-9) 



W is inverse transformed horizontally. If N = = 4, the one-dimensional inverse transform is performed as specified i 
subclause 8.6.2.3. If N = = 8, the inverse transform is specified by Equation 12-10, 



Z'= 



00 



(M-\)0 



07 



<A/-1)7 
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13 
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(12-10) 



The result is rounded 
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Z tj =sign(zv)[abs(zv)+2 8 -- , J»B Jhm 



(12-11) 



where B Mft = = 7 for 8x8 blocks, and fl**, = = 2 for 4x8 or 8x4 blocks. If N = = 4, the second one-dimensional inverse 
transform is performed as specified in subclause 8.6.2.3. If N = = 8, the second inverse transform is specified by ' 
Equation 12-12 below, 
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£70 



2 0</V-l) 



(12-12) 



After the second (vertical) transform in Equation 12-12 step the final reconstructed sample residual values X" shall be 
obtained as specified in Equation 8-59. 

Finally, the reconstructed sample residual values X" from Equation 8-59 are added to the prediction values P l} from 
motion compensated prediction or spatial prediction and clipped to the range of 0 to 255 to form the final decoded 
sample result prior to application of the deblocking filter as specified in subclause 8.6.3. 



12.4.4 



Modifications for the deblocking filter 



If ABT is used, the boundary strength shall be Bs = 0 for all 4x4 luma block edges inside an ABT block. The index into 
the threshold table (Table 8-3) is increased by Iqp- For ABT blocks, I QP depends on the sizes of the neighbouring blocks 
as specified in Table 12-4. I QP = 0 for non-ABT blocks. 

Table 12-4 - Iq P values 



Iqi> 


Block q 


4x4 


4x8 


8x4 


8x8 




4x4 


0 


1 


1 


2 


Block p 


4x8 


1 


2 


2 


3 


8x4 


1 


2 


2 


3 




8x8 


2 


3 


3 


3 



The index used to access the a-table, as well as the CO-table that is used in the default filter mode, is computed as: 
Index A = Clip3(0, 51, QP. V + Iqp + Filter_Offset_A) 

The index used to access the (J-table is computed as: 

IndexB = Clip3(0, 51, QP av + Iqp + Filter_Offset_B), 

with QP, V , Filter_OfFset_A, and Filter_Offset_B as specified in subclause 8.7.2. The values for the thresholds (a and p) 
are specified in Table 8-3. 



146 ITU-T Rec H.264 (2002 E) 

<XP 1 1 0064 1 A_1_> 



ISO/IEC FCD 14496-10 

12.5 ABT entropy coding 

12.5.1 ABT variable length coding 

12.5.1.1 Mapped Exp-Golomb entropy coding 

VXf k T T a ™ Cr ° bl ° ck types in l™* sl ™° and the intra_bIock_typeABT syntax elements in Inter slices are to Exp- 
Golomb codeword numbers as specified in Table 12-5. uc " p 

Table 12-5 - Assignment of Exp-Golomb codeword numbers for ABT syntax elements 



code_number 


mb_type 


intra_block_typeABT 


0 


ABTIntra_4x4 


4x4 


1 


ABTIntra_4x8 


4x8 


2 


ABTintra_8x4 


8x4 


3 


ABTIntra_8x8 


8x8 



12.5. 1.2 VLC entropy coding of ABT coefficients 
12.5.1.2.1 Decoding num_coeff_abt 

For ABT Intra blocks num_coeff_abt is specified. The structure of the codewords for num coeff and escaoe run 

specified ,n subclause 12.4.2.4, is indicated in Table 12-6. The info bits xi, i=0 to n can take value's Oor 1 CSCape - run ' 

Table 12-6 - Code structure for ABT num_coeff_abt and escape_run 



codeword 


length L 


1 xl xO 


3 


0 1 x2 xl xO 


5 


00 1 x3x2xl xO 


7 


0 00 1 x4x3x2xl xO 


9 


000 0 1 x5x4x3 x2xl xO 


11 



The value of num_coeff is specified as the code number of the decoded codeword. The code number is specified 
code_number = 2 (L * ,V2 - 4 + INFO 

For a codeword with info bits xi, /'=0 to n, INFO is specified as 

INFO = £x/-2' . 



(12-13) 



i=0 



(12-14) 



12.5.1.2.2 2D (level,run) symbols 

^Tawfl^ d6pendS ° n b '° ck ty P e - The structure of the cod <* * specified 

in Table 12-7 For al block types, the codewords w.th code numbers 0 to 59 are used, with code number 59 being the 
escape symbol. INFO is specified in Equation 12-14 y ° eing tne 

Table 12-7 - Code structure for ABT (level, run) symbols 



Block type 


Code structure 


length L 


code_number 


Intra 8x8, 8x4, 4x8, 4x4 
Inter 8x8 


1 xl xO 


3 


2^^-4-fINFO 


0 1 x2xl xO 


5 
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00 1 x3x2xl xO 


7 | 






0 00x4x3 x2xl xO 


8 \ 






1 xO 


2 






0 1 xl xO 


4 




Inter 8x4, 4x8 


00 1 x2xl xO 


6 


2 (Ufiy2 -2 + INFO 




0 00 1 x3 x2xl xO 


8 






0 000x4 x3 x2xl xO 


9 






1 


1 


0 




0 1 xO 


3 




Inter 4x4 


00 1 xl xO 


5 


2 in 1 + INFO 


0001 x2xl xO 


7 






000 0 I x3 x2xl xO 


9 






0 0000x4x3x2x1 xO 


10 


2 ( L+ iy2 _ , + INFO 



12^.1.23 Assignment of level and run to code numbers 

For positive level, the assignment of code numbers to run and level is specified in Table 12-9. Run and negative levels 
are assigned as follows 

code_number( -abs(level), run ) = code_number( abs(level), run ) + 1 . (12-15) 



12^.1.2*4 escape Jevel and escape_run 

The code structure for escape Jevel is specified in Table 12-8. The code number of a decoded codeword is 
code.number = 2 (L * 2y2 - 8 + INFO, 



(12-16) 



with INFO as specified in Equation 12-14. The assignment of code numbers to escape Jevel is specified as follows: 



if( ( code_number % 2) > 0 ) 

escapejevel = -(code_number/2) 
else 

escapejevel = code_number/2 



(12-17) 



The code structure for escape_run is specified in^Table 12-6. The value of escape_run is specified as the code number of 
the decoded codeword. The code number for escape_run decoding is specified in Equation 12-13. 

Table 12-8 - Code structure for escapejevel 



Codeword 


length L 


1 x2 xl xO 


4 


0 1 x3 x2 xl xO 


6 


00 1 x4x3x2xl xO 


8 


000 1 x5 x4x3x2xl xO 


10 


0 00 0 1 x6x5x4x3x2xl xO 


12 


0 0 0 0 0 1 x7 x6 x5 x4 x3 x2 xl xO 


14 
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000000 1 x8x7x6x5x4x3x2xl xO 


16 


0000000 1 X9x8x7x6x5x4x3x2xl xO 


18 



Table 12-9 - Assignment of Inter and Intra level and run to code numbers. 





1 J 1LCJ 




Intra ; 






le\ 


'el>C 


I 










level>0 ( QP < 26 ) 


level>0 (26 <= QP < 34 ) 


level>0 ( QP >= 


:341 




run 




1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


1 


2 


3 


4 


5 


6 


7 


0 


0 


1 


5 


13 


21 


31 


39 


47 


0 


2 


6 


8 


12 


18 


20 


0 


2 


8 


12 


18 


22 


30 


0 


4 


12 


20 


32 


42 


56 


1 




3 


15 


33 


51 


- 


- 


- 


4 


14 


24 


30 


38 


44 


49 


4 


16 


26 


36 


48 


56 


- 


2 


16 


30 


48 


- 


- 


- 


2 




7 


25 


53 


- 


- 


- 


- 


10 


28 


40 


50 


- 


- 




6 


24 


42 


- 




- 


- 


6 


24 


46 


- 


- 


- 


- 


3 




9 


35 












16 


34 


54 


- 


- 


- 


- 


10 


34 












8 


34 












4 


: 


11 


45 












22 


42 












16 


44 












10 


40 












5 


- 


17 


55 












26 


56 












20 


5 












14 


50 












6 




19 
























































7 


- 


23 
























































8 




27 




























































29 
























































10 




37 
























































11 




41 
























































12 




43 
























































13 




49 
























































14 




57 

























































12.5-2 ABT CABAC 

12.5.2.1 Fixed-length (FL) binarization for imWype 

Table 12-10 shows the binarization scheme used for decoding of macroblock types in I-slices. In case of mb_type = 6 for 
P slices or mb_type = 23 for B slices, intra_block_typeABT has to be decoded as additional information related to the 
chosen intra mode for these macroblock types. This shall be done in the same way as specified for mb_type in I slices. 



Table 12-10 - Binarization for macroblock type 



Slice type 


Code number for mb_type 


Binarization 


I slice 


0 (ABTIntra_4x4) 




0 














1 (ABTIntra_4x8) 


0 


0 














2 (ABTIntra_8x4) 


0 


1 














3 (ABTIntra_8x8) 


1 


1 















12.5.2.2 Context definition and assignment 

Table 12-11 provides the context identifier associated to the syntax element macroblock type. A detailed description of 
the corresponding context variables is given in the subsequent subclauses. For the syntax elements related to decoding of 
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transform coefficients, each of the context identifiers utilizes a separate set of ranges depending on whether the 
additional context categories 5-7 given in Table 12-14 are used, which is only the case if MbABTFlag=L 



Table 12-11 - Macroblock typ and associated context identifier 



Syntax element 


Context identifier 


Type 
of 

Binarization 


max_idx_ctx_id 


Range of 
context 
label 


Macroblock type 


ctx_mb_type_I_ABT 


Table 
10-16 


2 


0-4 


Transform coefficients 


ctx_cbp4 


-/- 


-/- 


75 - 94,267 - 274 


ctx_sig 




-/- 


95-155.275-319 


ctxjasc 


-/- 


-/- 


156 - 216, 320 - 346 


ctx_abs_level 


UEGO, 
UCoff=14 


2 


217-266, 347-376 



12.5.2.2.1 Assignment of context labels 

Tables 12-12 and 12-13 contain context identifiers along with their corresponding range of context labels. The 
association of context labels (modulo some offset) and bin numbers shows which context variable uses a fixed model and 
which one implies a choice of different models. The latter are characterized by those entries where a set of different 
context labels are given for a specific bin_.no (Table 12-12) or block type dependent context_category (Table 12-13); 
these are the context variables, which need to be specified further in the following subclauses 12.5.2.2.2 and 12.5.2.2.3. 



Table 12-12 - Context identifiers and associated context labels 



Context identifier 


Range of 
context 
label 


Offset for 
context label 


max_idx_ctx_id 


bin_no 


1 


2 


3 


4 


>5 


ctx_mb_type_I_ABT 


0-4 


0 


2 


0,1,2 


3.4 


-/- 


-/- 


-/- 


ctx_abs_level 
(context_category 5-7) 


347-376 


347+ 

1 O*(context_category— 5) 


2 


0to4 


5to9 


5to9 


5 to 9 


5 to 9 



Table 12-13 - Context identifiers and associated context labels (continued) 



Context 
identifier 


Offset (range) of 
context label for 
context_category 
5-7 


comexL_category of block_type 


0 


1 


2 


3 


4 


5 


6 


7 


ctx_cbp4 


267 (267 - 274) 


0-3 


4-7 


8-11 


12-15 


16-19 


-A 


0-3 


4-7 


ctx_sig 


275 (275-319) 


0- 14 


15-28 


29-43 


44-^6 


47-60 


0- 14 


15-29 


30-44 


ctx_last 


320 (320 - 346) 


0-14 


15-28 


29-43 


44-46 


47-60 


0-9 


10-19 


20-29 



12.5.2.2.2 C ntext definitions using preceding bin values 

For the context identifier ctx_mb_type_I_ABT, the choice of the model for the 2nd bin depends on the value of the first 
bin as specified in Equation 12-18: 

ctx_mb_typeJ^ABT[2] = (bl = = 0) ? 3: 4 (12-18) 
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Table 12-14 - Additional context categories for the different bl ck types 



block_type 


Maximum 
number of 
coefficients 


contex t_category 


Luma block for INTRA 8x8 mode 


64 


5:Luma-8x8 


Luma block for INTER 8x8 mode 


64 


Luma block for INTRA 8x4 mode 


32 


6:Luma-8x4 


Luma block for INTER 8x4 mode 


32 


Luma block for INTRA 4x8 mode 


32 


7:Luma-4x8 


Luma block for INTRA 4x8 mode 


32 



12.5.2.2.3 Additional context definitions for information related to transform coefficients 

As specified in subclause 9.2.2.4, three different additional context identifiers are used for conditioning of information 
related to transform coefficients. All these three types depend on context categories of different block types denoted bv 
the variable context_category. The definition of these context categories is given in Tables 9-24 and 12-14 Note that the 
context categories 5 -7 are on\y used in the case when MbABTFlag = 1 . The context identifiers ctx sigandctx last are 
related to the binary valued information of SIG and LAST; the definition of the related context variables includes an 
additional dependency on the scanning position scanning_pos within the regarded block: 



ctx_sig[scanning_pos] = Map_sig( scanning_pos), 



(12-19) 



ctxJast[scanning_pos] = Map_last(scanning_pos). 



(12-20) 



The definition of Map_sig and Map_last in Equations 12-19 and 12-20 depends on the block type. For context category 
0 - 4 the corresponding maps are given by - B 3 

Map_sig(scanning_pos) = Map_list(scanning_pos)=scanning_pos, if context _category = 0,..., 4, (12-21) 

ri e A r ™ nning -, P ° S dCn ° teS me P° sition re,ated to * e z 'g- za S scan. For context categories 5-7, which are only in use if 
MbABTFlag = 1, two cases are distinguished. In frame coding mode, where the zig-zag scan is used, Map sig and 
Mapjast are given by the definition in Table 12-15; for field coding mode, Map.sig and Map last related to the 
alternative scans are given in Tabel 12-16. In each case, the offset for the context category given in Table 12-17 has to be 
added for calculating the context label of each scanning position. 

For abs_level_ml, the decoding process is specified in subclause 9.2.2.4. 

Table 12-15 - Map_sig and Mapjast for zig-zag scanning order used for the additional ABT block sizes 8x8 8x4 

and 4x8 



Scanning 
position 


8 


x8 


Scanning 


8x8 


Scanning 


8x4 and 4x8 


Map_sig 


Map_last 


position 


Map__sig 


Map_last 


position 


1 Map_sig 


Mapjast 


0 


0 


0 


32 


7 


3 


0 


0 


0 


1 


I 




33 


6 


3 


1 


1 




2 


2 




34 


11 


3 


2 


! 2 




3 


3 




35 


12 


3 


3 


3 




4 


4 




36 


13 


3 


4 


4 




5 


5 




37 


1! 


3 


5 


5 




6 


5 




38 


6 


3 


6 


7* 




7 


4 




39 


7 


3 


7 


8 
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41 
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3 
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42 


14 


4 


10 


11 


2 


11 


4 


1 


43 


10 


4 


11 


9 


2 


12 


4 


1 


44 


9 


4 
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8 


2 


13 


4 


1 


45 


8 


4 


13 


6 


2 


14 


5 


1 


46 


6 


4 


14 


7 


2 


15 


5 


1 


47 


11 


4 


15 


8 
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16 


4 


2 


48 


12 


5 


16 


9 


3 


17 


4 


2 


49 


13 


5 


17 


10 


3 


18 


4 


2 


50 


11 


5 


18 


11 


3 


19 


4 


2 


51 


6 


5 


19 


9 


3 


20 


3 
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52 


9 


6 


20 


8 


4 


21 


3 


2 


53 


14 


6 


21 


6 


4 


22 


6 


2 


54 


10 


6 


22 


12 


4 


23 


7 


2 


V 55 


9 


6 


23 


8 


4 


24 


7 
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11 


7 


24 


9 


5 


25 


7 
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57 


12 


7 


25 


10 


5 


26 


8 


2 


58 


13 


7 


26 


11 


6 


27 


9 


2 


59 


11 


7 


! 27 


9 


6 


28 


10 


2 


60 


14 


8 


28 


13 


7 


29 


9 


2 


61 


10 


8 


29 


13 


7 


30 


8 


2 


62 


12 


8 


30 


14 


8 




7 


2 


63 






31 






e 12-16 -Mi 




tofap to/ for field-based scanning order used for the additional ABT block size* 
8x4 and 4x8 


Scanning 
position 


8x8 


Scanning 


8x8 


Scanning 


8x4 and 4x8 


Map_sig 


Mapjast 


position 


Map_sig 


Mapjast 


position 


Map_sig 


Mapjast 


0 


0 


0 


32 


9 


3 


0 


0 


0 


1 






33 


9 


3 


1 


1 




2 
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34 


10 


3 


2 


2 




3 
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35 


10 


3 


3 


3 




4 
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36 


8 


3 


4 


4 




5 


3 




37 


11 


3 


5 
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6 
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38 


12 


3 


6 
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7 
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39 


11 


3 


7 
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12.5.2.3 



Initialisation of context models 
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The initialization procedure for the context models is specified in subclause 9^ in .hie „.k i .u 

parameters for the additional context modelsin subclause^ 2.4 2 » specified ^ ,n,t,al,zallon 

Table 12-17 - Initialisation parameters for context identifier ctx_mb_type_I_ABT 



Context 
label 


ctx_nib_typeJLABT 


m 


n 


0 


-8 


53 


1 


2 


50 


2 


17 


20 


3 


2 


50 


4 


2 


50 



.1 1 00641 A_l_> 



ITU-T Rec. H.264 (2002 E) 



153 



ISO/IEC FCD 14496-10 

Tabic 12-18 - Initialisation param ters for context id ntifiers ctx_cbp4, ctx_sig, ctxjast, 

category 5-7 





Context category 5 




Context category 6 




Context ca 


tegory 7 


Context 
label 


l-slices 


P3-sIices 


Context 
label 


l-slices 


P.B 




Context 
laoei 


I-slic 


es 


P.B-sl 


ices 




in | 


n 


-l 


n 




m 


n 


m 


n 




m 


n 


in 


n 








ctx_cbp4 






> 


















267 


-4 


63 


-2 


61 


271 




63 


-3 


63 












268 


-1 


70 


-7 


75 


272 


-7 


74 


-15 


75 












269 


-7 


68 


-12 


70 


273 


-I 


70 


-13 


80 












270 


-5 


76 


-20 


86 


274 


-5 


76 


-21 


88 












ctx_sig 












275 




59 


-4 


44 


290 


-8 


69 


-3 


48 


305 


o 

-o 


oo 


_3 


49 


1 OTA 

27t> 


-7 


55 


-3 


34 


291 


-12 


67 


-5 


47 


306 


-11 


68 


-6 


47 


| 277 


-6 


58 


-2 


35 


292 


-11 


68 


-3 


47 


307 


-11 


64 


-3 


48 


I 278 


-7 


53 


-4 


33 


293 


-12 


63 


-7 


47 


308 


-11 


56 


.7 


46 


1 279 


-9 


52 


-5 


31 


294 


-12 


66 


-7 


48 


309 


-13 


63 


-6 


47 


280 


-5 


48 


-2 


31 


295 


-12 


66 


-3 


46 


310 


1 1 

- 1 1 


67 


.3 


45 


! 281 


-14 


59 


-7 


36 


296 


-7 


60 


-7 


48 


311 


-8 


63 


-8 


49 


{ 282 


-8 


53 




31 


297 


-12 


63 


-4 


4Z> 


312 j 


- 12 


65 


-6 


46 


283 


-10 


54 


-4 


33 


298 


-11 


64 


-2 




313 


-1 1 


63 


-3 


46 


284 


-5 


47 


4 


29 


1 299 


-14 


64 


-5 




314 


-13 


60 


-2 


45 


I 285 


-4 


43 


0 


29 


300 


-12 


57 


-8 


41 


315 


-11 


53 


-7 


45 


1 7ftfi 
1 ZOO 


-13 


56 


2 


31 


301 


-16 


57 


0 


35 


316 


-13 


52 


-2 


37 


1 287 


-9 


49 


0 


31 


302 


-9 


53 


5 


40 


317 


-8 


50 


0 


38 


Oftft 
i ZOO 


-8 


47 


5 


23 


303 


7 


54 


-3 


57 
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8 


52 


-3 


54 
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J Z07 


3 


44 


15 


28 


304 


2 


67 


5 


63 


319 


1 


67 


7 
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29 


17 


27 


'329 


9 


25 
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10 
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25 


25 


9 
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29 
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17 
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25 


25 


7 
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8 


24 


25 


7 
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9 


28 


24 


21 
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13 


22 


27 


12 
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15 


21 


26 


13 
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18 


22 


22 


28 
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18 


19 


24 


19 
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17 


25 


19 


I jZ*t 


19 


23 


23 


31 
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20 


22 


24 


24 
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25 


22 


21 


29 
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23 


23 


23 


36 
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25 


23 


25 


32 
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28 


23 


25 


33 
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17 


43 
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21 


30 


22 


36 
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22 


31 


13 


44 
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14 


41 


17 


49 
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22 
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22 


45 
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15 
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50 
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58 
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-3 
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10 
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Annex A 
Profile and level definitions 

(This annex forms an integral part of this Recommendation | International Standard) 

> 

A.l Introduction 

Profiles and Levels specify the capabilities needed to decode the coded data, and may be used to indicate interoperability 
points between individual decoder implementations. 

NOTC - This Recommendation | International Standard does not include individually selectable "options" at the decoder, as this 

would increase interoperability difficulties. 
Each Profile defines a set of algorithmic features and limits which shall be supported by all decoders conforming to that 
Profile. Note that encoders are not required to make use of any particular set of features supported in a Profile. 

Each Level defines a set of limits on the values which may be taken by the parameters of this Recommendation | 
International Standard The same set of Level definitions is used with all Profiles, but individual implementations may 
support a different Level for each supported Profile. For any given Profile, Levels generally correspond to decoder 
processing and memory capability, in units based on video decoding, rather than on specific implementation platforms. 

A.2 Requirements on video decoder capability 

Capabilities of video decoders conforming to this Recommendation | International Standard are specified in terms of the 
ability to decode video streams conforming to constraints Profiles and Levels specified in this Annex. For each such 
Profile, the Level supported for that Profile shall also be expressed. Such expression may be in the form of coded values 
equivalent to a specific Profile and specific Level from this Annex. 

Specific values are defined in this annex for the syntax elements profile Jdc and leveljdc. All other values of 

profile_idc and level^idc are reserved for future use by ITU-T | ISO/IEC. 

NOTE- Decoders should not infer that if a reserved value of profilejdc or leveljdc falls between the values specified in this 
Recommendation | International Standard this indicates intermediate capabilities between the specified profiles or levels, as there 
are no restrictions on the method to be chosen by ITU-T | ISO/IEC for the use of such future reserved values. 



A.3 Baseline profile 
A3.1 Features 

All decoders supporting the Baseline Profile shall be capable of decoding bitstreams which use the following features: 

a) I and P picture types 

b) In-loop deblocking filter 

c) Frame pictures with mb Jevel_af f = 0 

d) 1/4-sample motion compensation 

e) Tree-structured motion segmentation down to 4x4 block size 
0 VLC-based entropy coding 

g) Arbitrary slice order (ASO): In Baseline profile, the decoding order of slices within a picture may not follow the 
constraint that first_mb_in_slice is monotonically increasing within the NAL unit stream for a picture. 

h) Flexible macroblock ordering (FMO, maximum 8 slice groups): In Baseline profile, 
num_slice_groups_minus 1 <8. 

i) Redundant slices 

j) 4:2:0 Chrominance format 

Decoders supporting the Baseline Profile Level 2.1 and above shall also be capable of decoding bitstreams using: 
k) Field pictures. 

Conformance to the Baseline Profile is indicated by setting the syntax element profilejdc equal to 66. 
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A.3.2 



Limits 



All decoders supporting this Profile shall be capable of decoding bitstreams which: 

a) use 15 or fewer Reference Frames, 

b) have a compression ratio per picture of 4: 1 or greater, 

c) use 64 or fewer Picture Parameter Sets, and, 

d) use 16 or fewer Independent Sequence Parameter Sets. 



All decoders supporting the X Profile shall be capable of decoding bitstreams which use the following features: 

a) Bi-predictive slices 

b) SP and SI slices 

c) Data partitioned slices 

d) Weighted prediction 

e) AH features included in the Baseline Profile 

All video decoders supporting the X Profile shall also support the Baseline Profile. The Level number supported for the 
Baseline Profile shall not be less than the Level number supported for the X Profile. 

Conformance to the X Profile is indicated by setting the syntax element profile_idc equal to 88. 

A.4.2 Limits 

All decoders supporting this Profile shall be capable of decoding bitstreams which: 

a) use 15 or fewer Reference Frames, 

b) have a compression ratio per picture of 4: 1 or greater, 

c) use 64 or fewer Picture Parameter Sets, and, 

d) use 16 or fewer Sequence Parameter Sets. 

A.5 Main profile 
A.5.1 Features 

All decoders supporting the Main Profile shall be capable of decoding bitstreams which use the following features: 

a) Bi-predictive slices 

b) CABAC 

c) Weighted prediction 

d) Adaptive block-size transforms (ABT) 

e) All features included in the Baseline Profile except: 

1. Arbitrary Slice Order (ASO): In Main profile, the decoding order of slices within a picture shall follow the 
constraint that first_mb_in_slice shall be monotonically increasing within the NAL unit stream for a 



2. Flexible Macroblock Order (FMO): In Main profile, num_s!ice _groups_minusl shall be 

3. Redundant Slices 

Decoders supporting Level 2.1 and above shall also be capable of decoding bitstreams using: 

f) Interlaced pictures, frame/field adaptive at picture level and macroblock level. 
Conformance to the Main profile is indicated by setting the syntax element profilejdc equal to 77. 
A.5.2 Limits 

All decoders supporting this Profile shall be capable of decoding bitstreams which: 

a) use 15 or fewer Reference Frames, 

b) have a compression ratio per picture of 4: 1 or greater, 

c) use 64 or fewer Picture Parameter Sets, and, 

d) use 16 or fewer Sequence Parameter Sets. 



A.4 



X profile 



A.4.1 



Features 
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A.6 Level definitions 
A.6.1 General 

Level limits are expressed in units of whole luma macroblocks. If a particular picture height or width is not an exact 
multiple of a whole macroblock, that dimension shall be considered as rounded up to the next whole macroblock for the 
purposes of conformance with this subclause. 

The definition of support for a given Level is that any picture size/frame rate combination shall be decoded where the: 

a) sample processing rate (in whole macroblocks/second) is <= the Level limit given, and, 

b) picture size (Height * Width, in whole macroblocks) is <= the Level limit given, and, 

c) required reference memory is <= the Level limit given, and, 

d) maximum video bit rate of the bitstream is <= the Level limit given, and, 

e) required HRD/VBV buffer size is <= the Level limit given, and, 

0 horizontal and vertical motion vector range does not exceed the Level limits given, and, 

g) picture Height and picture Width (in whole macroblocks) are <= sqrt(LevelLimitMaxPictureSize * 8), and, 

h) frame rate is not greater than 172 Hz. 

The definition of each Level includes the requirements of all lower numbered Levels in Table A-l. Decoders supporting 
a given Level shall also be capable of decoding bitstreams using all lower numbered Levels. 

Note that display of decoded video is outside the scope of this Recommendation | International Standard; some decoder 
implementations may not include displays at all, and display limitations do not necessarily cause interoperability failures. 

"Picture size" means the total number of macroblocks in the complete picture (both even and odd fields if interlaced). 
A.6,2 Level limits 

Table A-l below gives the parameter limits for each Level. Conformance to a particular Level shall be indicated by 
setting the syntax element leveljdc equal to a value of ten times the level number specified in Table A-l. 

Table A-l - Level Limits 





Max 






Max 


Max 










Sample 


Max 


Reference 


Video 


HRD/VBV 






Minimum 




Processing 


Picture 


Memory 


Bitrate 


Buffer 


Horizontal MV 


Vertical MV 


luma Bi- 


Level 


Rate 


Size 


(1024 
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Range 


Range 


predictive 


# 


(MB/s) 


(MBs) 


bytes) 


bits/sec) 


(bits) 


(full pels) 


(full pels) 


block size 


1 


1 485 


99 


148.5 


64 


163 840 


[-2048,2047.751 


[-64,+63.75] 


8x8 


1.1 


2 970 


396 


891.0 


128 


327 680 


[-2048, 2047.75] 


[-128,+127.75] 


8x8 


1J2 


5 940 


396 


891.0 


768 


1 966 080 


[-2048, 2047.75] 


[-128,+ 127.75] 


8x8 


2 


11 880 


396 


891.0 


2000 


2 000 000 


[-2048, 2047.751 


[-128,+127.75] 


8x8 


2.1 


19800 


792 


1 782.0 


4000 


4000 000 


[-2048, 2047.75] 


[-256.+255.751 


8x8 


2.2 


20 250 


1 620 


3 037.5 


4 000 


4000 000 


[-2048, 2047.75] 


[-256.+25S.75] 


8x8 


3 


40 500 


1 620 


3 037.5 


8000 


8000 000 


[-2048, 2047.75] 


[-256,+255.75] 


8x8 


3.1 


108 000 


3 600 


6 750.0 


20 000 


20 000 000 


[-2048, 2047.75] 


[-512,+511.75] 


8x8 


3.2 


216 000 


5 120 


7 680.0 


20 000 


20 000 000 


[12048, 2047.75] 


[-512.+511.75] 


8x8 


4 


245 760 


8 192 


12 288.0 


20 000 


20 000 000 


[-2048, 2047.75] 


[-512,+511.75] 


8x8 


5 


491 520 


19 200 


28 800.0 


TBD 


max bps) 


[-2048,2047.75] 


TBD 


8x8 



Levels with non-integer Level numbers in Table A-l are refered to as "intermediate Levels". All Levels have the same 
status, but note that some applications may choose to use only the integer-numbered Levels. 

Informative subclause A.7 shows the effect of these limits on frame rates for several example picture formats. 
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A-6.3 Reference memory constraints on modes 

"Reference Memory" means the decoder memory pool which is used to store reference frames and post-decoder frame 
buffers. Note that P pictures require one reference frame, and B pictures require two reference frames; any remaining 
reference memory may be used either for multiple reference frames or for post-decoder frame buffers. 

Decoders shall support all bitstreams within a given Profile and Level where the required reference memory does not 
exceed the limit given for the Level. ! 

The reference memory required for a given mode shall be calculated as: 

bytes = PictureSize * NumberOfReferenceFrames * ChromaFormatParameter * 256 

The PictureSize parameter is in units of whole macroblocks. 

The parameter ChromaFormatParameter shall take values according to Table A-2: 

Table A-2 - ChromaFormatParameter values 



Chrominance 
Format 


ChromaFormatParameter 


monochrome 


1 


4:2:0 


1.5 


4:2:2 


2 


4:4:4 


3 



A.7 Effect of level limits on frame rate (informative) 

This subclause does not form an integral part of this Recommendation | International Standard. 



Level number: 
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t (macroblocks): 
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11380 
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491,520 


Max picture size (samples): 




25.344 


101.376 


101.376 


101.376 


202.752 


414.720 


414.720 921.600 


1.310.720 2.097.152 4.915.20 


Max san 


iples/second (1000s): 
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1.521 


3,041 
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5.184 


10.368 


27.648 


55,296 


62.915 
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MB 
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Format 
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SQCIF 


128 


96 


8 


6 


30.9 


61.9 


123.8 


172.0 


172.0 


172.0 


172.0 


172.0 


172.0 


172.0 


172.0 


QCIF 


176 


144 


11 


9 


15.0 


30.0 


60.0 


120.0 


172.0 


172.0 


172.0 


172.0 


172.0 


17Z0 


172.0 


QVCA 


320 


240 


20 


15 




9,9 


19.8 


39,6 


66.0 


67.5 


135.0 


172,0 


172.0 


172-0 


172,0 


SIF 


352 


240 


22 


15 




9.0 


18.0 


36.0 


60.0 


61.4 


122.7 


172.0 


172.0 


172.0 


172.0 


CIF 


352 


288 


22 


18 




7.5 


15.0 


30.0 


50.0 


51.1 


102.3 


172.0 


I7Z0 


172.0 


172.0 


2SIF 


352 


480 


22 


30 










30.0 


30.7 


61.4 


?63.6 


I72.Q 


172.Q 


172.Q 


HHR 


352 


576 


22 


36 










25.0 


25.6 


51.1 


136.4 


172.0 


172.0 


172.0 


VGA 


640 


480 


40 


30 












16.9 


33.8 


90.0 


172.0 


172.0 


172.0 


4SIF 


704 


480 


44 


30 












15.3 


30,7 


81.8 


163.6 


172.0 


172.0 


NTSCSD720 


480 


45 


30 












15.0 


30.0 


80.0 


160.0 


172.0 


172.0 


4CIF 


704 


576 


44 


36 












12.8 


25.6 


68.2 


136.4 


155.2 


172.0 


PALSD 


720 


576 


45 


36 












J2.£ 


25.0 


66,7 


J33.3 


151.7 


172.0 


SVGA 


800 


600 


50 


38 
















56.8 


113.7 


129.3 


172.0 


XGA 


1024 


768 


64 


48 
















35.2 


70.3 


80.0 


160.0 


72Qp 


1280 


720 


9P 


45 
















30,0 


60.0 


68.3 


136,5 


4VGA 


1280 


960 


80 


60 


















45.0 


51.2 


102.4 


SXGA 


1280 


1024 


80 


64 


















4Z2 


48.0 


96.0 


1<»IF 


1498 


960 


88 


60 




















46,5 


93.1 


I6CIF 


1408 


1152 


88 


72 




















38.8 


77.6 


4SVGA 


1600 


1200 


100 


75 




















32.8 


65.5 


1080i 


1920 


1080 


120 


6? 




















30.1 


60? 


2KxlK 


2048 


1024 


128 


64 




















30.0 


60.0 


4XGA 


2048 


1536 


128 


96 






















40.0 


NJVGA 


2560 


1920 


160 


120 






















25.6 



Note 1 This is a variable-picture-size specification. The specific picture sizes in this table are illustrative examples only. 

Note 2 XGA is also known as (aka) XVGA, 4SVGA aka UXGA, I6XGA aka 4Kx3K, HHR aka 2CIF aka 1/2 Dl aka 
1/2ITU-RBT.601. 

Note 3 Frame rates given are correct for progressive scan modes, and for interlaced if "MB High" column value is 
even. 
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Annex B 
Byte stream format 

(This annex forms an integral part of this Recommendation | International Standard) 

B.l Introduction 

This annex defines a byte stream format specified for use by systems that transmit some or all of the NAL unit stream as 
an ordered stream of bytes or bits within which the locations of NAL unit boundaries need to be identifiable from 
patterns in the data, such as ITU-T Recommendation H.222.0 | ISO/IEC 13818-1 systems or ITU-T Recommendation 
H.320 systems. For bit-oriented transmission, the network bit order for the byte stream format is defined to start with the 
MSB of the first byte, proceed to the LSB of the first byte, followed by the MSB of the second byte, etc. 

The byte stream format consists of a sequence of byte_stream_unit( ) structures. Each byte_stream_unit( ) contains one 
start code prefix (SCP) and one nal_unit( ). Optionally, at the discretion of the encoder, the byte_stream__unit( ) may also 
contain additional "stuffing" zero-valued bytes as specified in this clause. 

There are two types of start code prefixes: 

- A short SCP, consisting of one byte having the value zero (0x00) followed by one byte having the value 
one (0x01), and 

— A long SCP, consisting of two bytes having the value zero (0x00) followed by one byte having the value 
one (0x01). 

The long SCP provides a mechanism for decoder byte-alignment recovery in the event of loss of decoder 
synchronization. ' 

B*2 Byte stream NAL unit syntax 



byte_stream_unit() { 


Category 


Mnemonic 


while ( next__bits( 16 ) != 0x0001 && next_bits( 24 ) != 0x000001 ) 






zero_byte 




f(8) = 0x00 


if( next_btts() = = 0x000001 ) 






zero_byte 




f(8) = 0x00 


zero_byte 




f(8) = 0x00 


one_byte 




f(8) = 0x01 


nal_unit( ) 






} 







B3 Byte stream NAL unit semantics - 

The order of byte stream NAL units in the byte stream shall follow the decoding order of the NAL units contained in the 
byte stream NAL units. 

zerojbyte is a single byte (8 bits) having the value zero (0x00). Optionally, at the discretion of the encoder, the 
beginning of a byte_stream_unit() may contain more zero_byte syntax elements than required in this subclause. 

The minimum required number of zero_byte syntax elements depends on the nal_unit_type as defined in Table 7-1, in 
order to ensure the use of the long SCP for certain nal_unit_type values. This ensures use of the long SCP in the 
byte_stream_unit() for these nal_unit() structures. The use of the long SCP for nal_unit() structures with other values of 
nal_unit_type is optional. At least two zero_byte syntax elements shall be present in each byte_stream_unit() for the 
following values of nal_unit_type: 

- 0x06: Supplemental enhancement information, 

- 0x07: Sequence parameter set, 

- 0x08: Picture parameter set, and 
0x09: Picture delimiter. 
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one_byte is a single byte (8 bits) having the value one (0x01). A sequence of two zero.byte syntax elements followed 
by a one_byte is a long SCP, and one zero.byte followed by a one.byte is a short SCP. elements tollowed 

The use of the byte sequence 0x00 0x02 is reserved for future use by ITU-T | ISO/IEC. 
B.4 Decoder byte-alignment recovery (informative) 

This subclause does not form an integral part of this Recommendation | International Standard. 

If the decoder does not have byte alignment with the encoder's byte stream, the decoder can examine the incoming hit 
EHlS'h! r- J PfT,,' 0000 ^ 00000000 00000001 ' (23 zero-valued bits fiZ^TS££ 

Sl^JE^d!^*- Pattem ^ f ' rSt ^ ° f 3 Wh ° ,e - ~he diS 

SSSiSoSS^ * 11,6 ^ eXamine iRCOming bytE Stfeam for * e *>y« sequences 0x00 

S^T^Z^M 01 iS 1,115 rePreSCn,S 3 " PreVi ° US by,C W3S ° X00 ' the SCP * * SCP. 

diagram 6 ™" ™° * dB °° -Br the 0x03 aS shown in the rbsp_extrac«o„() syntax 

SSSS bytC aHgned ' ^ thUS ^ n ° ^ ^ bi, ° riented b »* detection 

S°^nnn h m yt f ! ,ignmem ^T^™ P rocedure described in this subclause is equivalent to searching a byte sequence 
the end of a X ?CP^dfh at rT ahgnmen ' P? S * on - Detec «"8 «■* P^rn indicates that the next non-fero b^SSZ 
the end of a SCP, and the first non-zero bit in that next non-zero byte is the last bit of an aligned byte. 
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Annex C 
Hypothetical Reference Decoder 

(This annex forms an integral part of this Recommendation | International Standard) 

> 

CA Hypothetical reference decoder and buffering verifiers 

The hypothetical reference decoder (HRD) represents a set of normative requirements on coded bitstreams or packet 
streams These constraints must be enforced by an encoder, and can be assumed by a decoder or multiplexor to be true- 
It is possible to verify the conformance of a bitslream or packet stream to the requirements of this subclause by 
examining the bitstream or packet stream only. 

This subclause defines the normative requirements of the HRD. Subclause C.2 provides additional information that is 
important for a full understanding of the HRD operation. 

Two types of streams may be subject to the HRD requirements of this Recommendation | International Standard; a 
stream of VCL NAL Units and a bitstream. Figure C-l shows how each of these is constructed from the RBSP. In other 
words, a given set of HRD parameters may pertain to the VCL data only or to the multiplexed combination of VCL and 
NAL. This is signalled through video usability information (subclauses E.2 and E.3). 



Parameter Set RBSP (7.3.2.1-2) — 
SEI Message RBSP (7.3.2.3, Annex D) — 
Picture Delimiter RBSP (7.3.2.4) — 
Video Usability Information (Annex E)^ 



EBSP Encapsulation 
Start Code Emulation Prevention 
NAL Unitization 



NAL Unitizer 



NAL NAL Units (7.3.1, 8.1) 



Bitstream 
Adaptation 
(Annex B) 



Slice Layer RBSP (7.3.2.6) 
Data Partition RBSP (7.3.2.7) 
Filler Data RBSP (7.3.2.5) 
RBSP Trailing Bits (7.3.2.8) 



NAL Unitizer 



EBSP Encapsulation 

Start Code Emulation Prevention 

NAL Unitization 



Legend 




VCL HRD Conformance Point 
— ix NAL HRD Conformance Point 



Figure C-l - Structure of Byte streams and NAL unit streams and HRD Conformance Points 



The HRD can contain any combination of the following buffering verifiers, as shown in Figure C-2: 

• One or more pre-decoder buffers, each of which is either variable bit rate (VBR) or constant bit rate (CBR) 

• At most one reference and post-decoder buffer attached to the output of one of the pre-decoder buffers 
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Figure C-2 - HRD Buffer Verifiers 



The multiple buffering verifiers exist because a bit stream or packet stream may conform to multiple pre-decoder buffers 
as detailed in subclause C.2.2. * 

All the arithmetic in this annex is done with real values, so that no rounding errors can propagate. For example the 
number of bits in a pre-decoder buffer just prior to or after removal of a transmitted picture is not necessarily an integer 
Furthermore, while conformance is guaranteed under the assumption that all frame-rates and clocks used to generate the 
bitstream match exactly the values signalled in the bitstream, each of these may vary from the signalled or defined value. 

This hypothetical reference decoder uses two time bases. One time base is a 90 kHz clock, and is only in operation for a 
short time after the reception of a Buffering Period SEI message. The second time base uses the num_unitsjn_tick and 
time_scale syntax in the Sequence Parameter Set to derive the time interval between picture removals from the buffers 
(and in some cases between picture arrivals to the pre-decoder buffer). 

In the following description, let t c = num_units_in_lick * time_scale be the clock tick associated with the second clock 
The clock tick is a time interval no larger than the shortest possible inter-picture capture interval in seconds Also let 
be[t] and te[b] be the bit equivalent of a time t and the time equivalent of a number of bits b f with the conversion factor 
being the buffer arrival bit rate. 

The following statements are normative requirements on the composition of a conforming bitstream If multiple 
sequence parameter sets pertain to the bit stream or packet stream, they must contain consistent HRD information In the 
case that any HRD buffers are signalled in the sequence parameter set(s), then the following rules dictate the insertion of 
SEI messages in the bit stream or packet stream. 

1. At each decoder refresh point (IDR, ODR or GDR), a buffering period SEI message shall follow the last 
NAL Unit of the last picture Before a decoder refresh and precede the first NAL Unit of the first picture 
after the decoder refresh. Note that in the case of an IDR, this SEI message will precede the indication of 
the decoder refresh point 

2. An HRD picture SEI message must follow the last NAL Unit of each picture and precede the first NAL 
Unit of the next picture. Each of these SEI messages pertains to the picture that follows it. 

C.1.1 Operation of VCL video buffering verifier (VBV) pre-decoder buffer 

This specification applies independently to each pre-decoder buffer VUI sequence parameters within the seauence 
parameter set. ^ 



C. 1.1.1 Timing of bitstream or packet stream arrival 

The buffer is initially empty. The first bit of the first transmitted picture begins to enter the buffer at initial arrival time 
U0)-0 at the bit rate bit_rate[k] associated with the pre-decoder buffer (see subclause 8.3.3). - The last bit of the first 
transmitted picture finishes arriving at final arrival time 
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r af (0) = b(0) + bit_rate[k], 



(C-l) 



where b(n) is the size in bits of the n-th transmitted picture. The final arrival time for each picture is always the sum of 
the initial arrival time and the time required for the bits associated with that picture to nter the pre-decoder buffer: 

r al (n) = an) + b(n) + bit_rate[k]. (C-2) 

For each subsequent picture, the initial arrival time of picture n is the later of t a /n-J) and the sum of all preceding 
pr _dec_removaI_delay times, as indicated in Equation C-3. 

n-l 

fai(")= max{ t M ^n-l\ t c x £ pre_dec_removal_delay(m) } (C-3) 

m=0 



See subclauses C.2.5 and C.3.5 for the syntax and semantics of the pre_dec_removal_deIay times. When the encoder is 
producing a bit rate lower than the bit rate associated with a pre-decoder buffer, this rule may delay the entry of some 
pictures into the pre-decoder buffer, producing periods during which no data enters. 

C.l.1.2 Timing of coded picture removal 

For the first picture and ail pictures that are the first complete picture after receiving a buffering period SEI message, the 
coded data associated with the picture is removed from the pre-decoder buffer at a removal time equal to the following: 

t^O) = irutial_pre_dec_removal_delay 90000 (C-4) 

r » 

where iniual_pre_dec_removal_delay is the pre-decoder removal delay in the buffering period SEI message. 

After the first picture is removed, the buffer is examined at subsequent points of time, each of which is delayed from the 
previous one by an integer multiple of the clock tick r c . 

The removal time tj/i) of coded data for picture n is delayed with respect to that of picture n-7; the delay is equal to the 
time indicated in the pre_dec__removal_delay syntax element present in the HRD picture SEI message. 

tf(n) = tj[n-l) + t c xpre_dec_removal_delay(n) (C-5) 



At this time, the coded data for the next transmitted picture is removed from the pre-decoder buffer. 

In the case that the amount of coded data for picture n, b(n), is so lar§e that it prevents removal at the computed removal 
time, the coded data is removed at the delayed removal time, t rM (n, m ), given by 

tUn,m*) = t T (0) + t c x m\ (C-6) 



where m is such that f r , d (n, m*-/) < t^n) < t fM (n, m\ This is an aspect of low-delay operation (see subclause C.2.1.2). 
This delayed removal time is the next time instant after the final arrival time t^n) which is delayed with respect to t^O) 
by an integer multiple of t c . 

C.l.1.3 Conformance constraints on coded bitst reams or packet streams 

A transmitted or stored stream of coded data conforming to this Recommendation | International Standard fulfils the 
following requirements. 

• Removal time consistency. For each picture, the removal times t&ri) computed using different buffering periods 
as starting points for conformance verification shall be consistent to within the accuracy of the two clocks used 
(90 kHz clock used for initial removal time and t c clock used for subsequent removal time calculations). This 
can be ensured at the encoder by computing the pre-decoder removal delay (initial_pre_dec_removaLdelay) for 
a buffering period SEI message from the arrival and removal times computed using Equations C-3 and C-5. 
Any small deviations between the values computed in the different ways shall not cause violation of any of the 
following constraints. 

• Underflow and Overflow Prevention. The buffer must never overflow or underflow. 

NOTE - In terms of the arrival and removal schedules, this means that, with the exception of some pictures in low-delay mode that 
are described below, all bits from a picture must be in the pre-decoder buffer at the picture's computed removal time tr(n). In other 



1 64 ITU-T Rec H.264 (2002 E) 

MSDOCID: <XP 1 1 00641 A_l_> 



ISO/IEC FCD 14496-10 

words, its final arrival time must be no later than its removal time: taf(n) < tr(n). Further, the removal time tr(n) must be no later 
than the time-equivalent of the buffer size te|pre_dec_buffer__size[k]]. Note that this prevents overflow. 

* BiZ Picture Removal Time. Overflow Prevention and Re synchronisation of Underflow Prevention If the 
final arrival time t^n) of picture n exceeds its computed removal time / f (n), its size must be such that it can be 
removed from the buffer without overflow at t TJi (n,m*) as defined above. 

* Constant Bit Rate Constraint. If vbr_cbr_nag[k] = = I, data shall arrive continuously at the input to the pre- 
decoder buffer. This is equivalent to ensuring that > t c x ]T pre_dec_removal_delay(m). 

* T [ me Burton Constraint, For each picture immediately preceding a Buffering Period SEI message the sum 
of pre-decoder removal delays from the start of the sequence up to that point of time shall be no further from the 
accumulated sequence duration as represented by the sum of prev_buf_period_duration than the 
removal_time_tolerance in the relevant sequence parameter set. 

If the picture immediately preceding the Buffering Period SEI message is the n-th picture in transmitted order 
and the buffering period SEI message is the A-th such message, then this constraint amounts to the following: ' 

I X, P r e-dec_removal_delay(m) - ^ prev_buf_period_duration(m) | < removal_time_tolerance (C-7) 



' Maximum Decoder Frame Rate. The interval between consecutive removal times shall not be lower than the 
minimum picture interval, defined as the inverse of the maximum picture rate (See A.5.1). 
i > 
C.1.2 Operation of the post-decoder buffer verifier 

C.l.2.1 Arrival timing 

A reconstructed picture is added to the post-decoder buffer at the same time when the corresponding coded picture is 
removed from the pre-decoder buffer. 

C.l.2.2 Removal timing 

Data is not removed from the post-decoder buffer during a period called the initial post-decoder buffering period The 
period starts when the first picture is added to the post-decoder buffer. 

When the initial post-decoder buffering period has expired, the playback timer is started from the earliest display time of 
the pictures residing in the post-decoder buffer at that time. 

A picture is virtually displayed when the playback timer reaches the scheduled presentation dme of the picture. 

A picture memory is marked unused in the post-decoder buffer when it is virtually displayed and when it is no loneer 
needed as a reference picture. 6 

C. 1.2.3 Conformance constraints 

The occupancy of the post-decoder buffer shall not exceed the default or signalled buffer size. 
Each picture shall be available in the post-decoder buffer before or on its presentation time. 

C.2 Informative description of the HRD 

Subclause C.l contains the normative requirements imposed by a set of buffering verifiers. This subclause provides 
explanatory text describing in more detail the operation and capabilities of these buffers. 

An HRD represents a means to communicate how the bit rate is controlled in the process of compression The HRD 
contains a pre-decoder buffer (or VBV Buffer) through which compressed data flows with a precisely specified arrival 
and removal timing, as shown in Figure C-3. An HRD may be designed for variable or constant bit rate operation, and 
for low-delay or delay-tolerant behavior. The HRD described in this document handles all cases 
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Figure C-3 - A Hypothetical Ref rence Decoder 



Compressed data representing a sequence of coded pictures flows into the pre-decoder buffer according to a specified 
arrival schedule. All compressed bits associated with a given coded picture are removed from the pre-decoder buffer by 
the instantaneous decoder at the specified removal time of the picture. 

The pre-decoder buffer overflows if the buffer becomes full and more bits are arriving. The buffer underflows if the 
removal time for a picture occurs before all compressed bits representing the picture have arrived. 

C.2.1 Constrained arrival time leaky bucket (CAT-LB) model 

The hypothetical reference decoder (HRD) is a mathematical model of a decoder and its input buffer. The &-th pre- 
decoder buffer of the HRD is characterized by the pre-decoder peak rate bit_rate[/c] (in bits per second), the buffer size 
pr _dec_buffer_size[&] (in bits), the sequence initial pre-decoder buffer removal delay (in seconds), as well as picture 
removal delays for each picture. The first three of these parameters represent levels of resources (transmission capacity, 
buffer capacity, and delay) used to decode a bitstream. 

The term "leaky bucket" arises from the analogy of the encoder as a system that "dumps" water in discrete chunks into a 
bucket that has a hole in it The departure of bits from the encoder buffer corresponds to water leaking out of the bucket 
Here, the decoder buffer is described, which has an inverse behaviour where bits flow in at a constant rate, and are 
removed in chunks. 

The leaky bucket described here is called a constrained arrival time leaky bucket because the arrival times of all pictures 
after the first are constrained to arrive at the buffer input no earlier than the difference in hypothetical encoder processing 
times between that picture and the first picture. In other words, if a picture is encoded exactly seven seconds after the 
first picture was encoded, then its bits are guaranteed not to start arriving in the buffer prior to seven seconds after the 
bits of the first picture started arriving. It is possible to know mis encoding time difference because it is sent in the 
bitstream as the picture removal delay. 

C.2.1.1 Operation of the CAT-LB HRD 

The HRD input buffer has capacity pre_dec_buffer_size[*] bits. Initially, the buffer begins empty. The lifetime in the 
buffer of the coded bits associated with picture n is characterized by the arrival interval {*»</*), t^ji)} and the removal 
time t&n). The end-points of the arrival interval are known as the initial arrival time and the final arrival time. 

At time tjifi) = 0, the buffer begins to receive bits at the rate bit_rate[/:]. The removal time t^ft) for the first picture is 
computed from the pre-decoder removal delay initial jre_dec_removal_delay (see Buffering Period SEI Message) 
associated with the buffer by the following: 

t£Q) = 90,000 x initial_pre_dec_rennoval_delay. (C-8) 



Removal times 1,(1), t^2\ r r (3), for subsequent pictures (in transmitted order) are computed with respect to f r (0), as 
follows. Let the clock tick t c be defined by 

t c = num_jinits_in_tick + time_scale (C-9) 



For instance, if time_scale = 60,000 and num_units„in_tick = 1,001, then 

t c = 1,001 - 60,000 = 16.68333... milliseconds. (C-10) 



In the HRD picture SEI message for each picture, there is a pre_dec_removal_delay syntax element This indicates the 
number of clock ticks to delay th removal of picture n after removing picture n-7. Thus, th removal time is simply 

*r(«) = t^n-l) + t c x pre_dec_removal_delay(/i)(C- 11) 
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(C-12) 



The calculation of arrival times is more complex, because of the arrival time constraint. The initial arrival time of 
picture n is equal to the final arrival time of picture «-/, unless that time precedes the earliest arrival time, computed by 



n 

'ai,eajiiest(/0 = h x ^ [pre__dec_removaLdelay(m)] 



(C-13) 



Let b{n) be the number of bits associated with picture n. The duration of the picture arrival interval is always the time- 
equivalent of the picture size in bits, at the rate bit_rate[k]. 



'•<«) - /ai(/j) s te[b(n)] = b(n) + bit_rate[k] 



(C-14) 



Figure C-4 demonstrates a segment of the pre-decoder buffer fullness plot for a CAT-LB with the parameters given in 
Table C-l and picture sizes given by the first column of Table C-2. Note that Table C-2 lists for each picture the values 
for many times of interest in the buffering process. In addition to quantities defined above, the second column of Table 
C-2 conta.ns i c , which represents a hypothetical encoding time equal to the earliest possible initial arrival time of the 
picture 

Table C-l - Attributes of an example CAT-LB HRD 



Attribute 


Value 


Units 


ume_scale 


I 


units per second 


num_units_in_tick 


1 


units per tick 


bit_rate 


1000 


bits per second 


pre_dec_buffer_size 


10 


bits 


initiaLdelay 


10 


seconds 



Table C-2 - Picture sizes, and encoding, arrival and removal times for the example CAT-LB HRD 



b 


te 


tai 


taf 


tai-te 


U 




tr-te 


5,000 


0 


0 


5 


0 


10 


10 


10 


1,000 


1 


5 


6 


4 


11 


6 


10 


1,000 


2 


6 


7 


4 


12 


6 


10 


1,000 




7 


8 


4 


13 


6 


10 


1,000 


4 


8 


9 


4 


14 


6 


10 


1,000 


5 


9 


10 


4 


15 


6 


10 


500 


6 


10 


10.5 


4 


16 


6 


10 


500 


7 


10.5 


11 


3.5 


17 


6.5 


10 


500 


8 


11 


11.5 


3 


18 


7 


10 


500 


9 


11.5 


12 


2.5 


19 


7.5 


10 


500 


10 


12 


12.5 


2 


20 


8 


10 


500 


11 


12.5 


13 


1.5 


21 


8.5 


10 


500 


12 


13 
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Buffer Fullness vs. Time 
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Figure C-4 - Buffer fullness plot for example HRD in Table C-2 with picture sizes given in Table C-3 



As can be seen from Table C-2, the initial picture is large, and is followed by five pictures at exactly the buffer arrival 
rate R. This is followed by twelve pictures at half the rate, four pictures at three times the rate and one picture at twice 
the rate. Following this are two segments with pictures at 30% and 50% of the rate, respectively. In Figure C-4, the time 
interval from 10 seconds to 18 seconds illustrates the behaviour when the bit rate is constant and at or below the rate /?. 
In fact, whenever the arrival bit rate remains less than R for a time, the lower points of the fullness curve will not change. 
Further, the fullness at the peak in such a segment will be proportional to the fraction of the peak rate being consumed by 
the pictures. From seconds 18 to 28, we see the temporary effect of an increase in arrival rate to above R. Once those 
large pictures start to exit the buffer, the bit rate of pictures leaving the buffer exceeds R, and the fullness decreases. This 
process terminates at second 32, when the big pictures have exited and the series of smaller pictures starts entering the 
buffer. During seconds 36-43, the 30% peak rate pictures are entering and leaving the buffer, and during seconds 43-52, 
30% peak rate pictures are leaving while 50% peal rate pictures are entering. Hence the buffer fullness rises. Once 50% 
peak rate pictures begin to leave, the fullness stabilizes at 50% full. Note that this pre-decoder buffer stabilizes at a 
fullness that is proportional to the ratio of the short-term average bit rate to the arrival bit rate, rather than at 100%.. 

In general, the curve of buffer fullness vs. time is given by the following expression: 



BF (0 = 2 UMn) < t < t T (n))xb(n) + /(/ ai (n) < t < f^x^W)] 



(C-15) 



This expression uses indicator functions /(-) with time-related logical assertions as arguments to sum only those pictures 
that are completely in the buffer at time /, plus the appropriate portion of the picture currently entering the buffer, if one 
is. The indicator function I(x) is 4 T if jc is true and 4 0' otherwise. 

C.2.1.2 Low-delay operation 

Low-delay operation is obtained by selecting a low value for the initial pre-decoder removal delay. This results in true 
low delay through the buffer because, under normal operation, no removal delay (/ r (n)-f ai (n)) can exceed the initial 
removal delay f r (0). To see this, consider that the maximum removal delay for picture n occurs when the initial arrival 
time is equal to the earliest arrival time. Therefore, the maximum removal delay is given by t&n)- t^^Jjt). But, 
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n 

t£n) = tXO) + 'c *X [pre_dec_removal_delay(m)], 



and 



^^ariicst(n) = r c *X [pre^dec^removaLdelayCm)], 



so 

tfn) - W«e*(") = ^ (C-16) 
Thus setting an initial low delay creates a steady-state low-delay condition. 

However, in low-delay operation, it is useful to be able to process the occasional large picture whose size is so large than 
that it cannot be removed by its indicated removal time. Such a large picture can arise at a scene change, for example. 
This would ordinarily lead to an "underflow" condition. When a large picture is encountered, the rules for removal are 
relaxed to prevent this. The picture is removed at the delayed removed time, / r .id(*, given by 

/r,w(n, rn) = t£0) + t c x m\ (C- 1 7) 

where m is such that / r , w (n, m*-7) < ^(n) + te[b(n)) < t tM (n t m*). Note that the buffer must be large enough that this 
large picture can be accommodated without Overflow. Immediately after such a picture is received the removal time of 
the next picture must be such that low-delay operation is resumed. An encoder can facilitate this by skipping a number 
of pictures immediately after the large picture, if necessary. 

C.2.1.3 Bitstream / packet stream constraints 

The buffer must not be allowed to underflow or overflow. Furthermore, all pictures except the isolated big pictures must 
be completely in the buffer before their computed removal times. Isolated big pictures are allowed to arrive later than 
th ir computed removal times, but must still obey the overflow constraint. In CBR mode, there must be no gaps in bit 
arrival. 

C.2.1.3.1 Underflow 

The underflow constraint, BF(t) > 0 for all t, is satisfied if the final arrival time of each picture precedes its removal 
time. 

Un) < t T (n) (C-18) 

This puts an upper bound on the size of picture n. The picture size can be no larger than the bit-equivalent of the time 
interval from the start of arrival to the removal time. 

b(n) < beMn) - Un)] (C- 1 9) 

Since the initial arrival time r«i(n) is in general a function of the sizes and removal delays of previous pictures, the 
constraint on b(n) will vary over time as well. 

C.2.1.3.2 Overflow 

Overflow is avoided provided the buffer fullness curve BF(t) never exceeds the buffer size B. 

The constraints that the initial pre-decoder removal delay must be no larger than the time-equivalent of the buffer size, 
t/fi) < te(B\ and that under normal operation no removal delay can exceed the initial one guarantee that no overflow 
occurs in normal operation. To avoid overflow of an isolated big picture, the picture size is constrained by 

b(n) <be[B-Un)] (C-20) 
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C.2.1.33 Constant bitrate (CBR) operation 

The CAT-LB model operates in constant bit rate mode if one further constraint is applied - that data must constantly 
arrive at the input of the buffer. This ensures that the average rate is equal to the buffer rate R. This model behaves like 
an MPEG-1 CBR model with variable frame rate. This condition is ensured if the final arrival time of picture n is no 
earlier than the earliest initial arrival time of picture n+1. 

n 

r af (n) > r^eariic^/i+Z) = t c x^T [pre_dec_removal_delay(m)] (C-21) 

m=l 

This time constraint puts a lower bound on b(n). 
C.2.1.4 Rate control considerations 

An encoder employs rate control as a means to constrain the varying bit rate characteristics of the coded bitstream or 
packet stream in order to produce high quality coded pictures at the target bit rate(s). A rate control algorithm may target 
a variable bit rate (VBR) or a constant bit rate (CBR). It may even target both a high peak rate using a VBR scheme and 
an average rate using a CBR scheme. Further, as shown in subclause C.2.2, multiple VBR rates can be targeted. 

Rate control must ensure conformance with the pre-decoder buffers. This is related to the first goal of rate control, but is 
not necessarily the same. In this subclause, the way the pre-decoder buffers influences rate control is discussed. In a 
VBR pre-decoder buffers, the buffer must not overflow or underflow, but gaps may appear in the arrival rate. In order to 
meet these constraints, the encoder must ensure that for all t 7 the following inequalities remain true: 

0 < BF(t) < B t for all /. (C-22) 

Using Equation D-14, this becomes: 

0 < £ U(t^n) < t < t t (n))xb(n) + Ifafa) < t < t^n))xbe(t-t^n))] < for all /. (C-23) 

n 



The buffer fullness B(t) is a piecewise non-decreasing function of time, with each non-decreasing interval bounded by 
two consecutive removal times. Therefore, it is sufficient to guarantee the conformance at the interval endpoints; i.e. at 
the removal times. In particular, if underflow is prevented at the start of an interval (just after removal of a picture), it is 
completely prevented. The same holds for overflow at the end of the interval, just prior to picture removal. Therefore, 
the points of interest are the removal times. In the buffer at t r \n) and contributing to Equation C.22 are picture n and 
possibly some additional pictures up to picture m>n (with the last picture possibly only partially in the buffer). All 
pictures earlier than picture n have been removed. At f r + (n), picture n has been removed. 

Thus when encoding picture «, one rate control task is to allocate bits to picture n and the others in the immediate future 
in such a way that overflow is prevented at f/(n), and underflow is prevented at r r + (/i). Most immediately, as long as b(ri) 

is small enough so that te[b(n)] < t T (n) - f ai (/i), both overflow at t x \n) and underflow at r/(n) are prevented. This is 
usually a very high limit, and a rate control method will most likely further limit b(n) through its bit allocation process. 

C.2.2 Multiple leaky bucket description 

C.2.2.1 Schedule of a bitstream 

The sequence of removal time and picture size pairs {Wn), b(n)), n=0,V,...} is called die schedule of a bitstream. The 
schedule of a bitstream is intrinsic to the bitstream, and completely characterizes the instantaneous coding rate of the 
bitstream over its lifetime. Although a bitstream may conform to VBVs with different peak bit rates and different pre- 
decoded buffer sizes, the schedule of the bitstream is independent of the VBV. 

C.2.2.2 Containment in a leaky bucket 

A leaky bucket with leak rate R u bucket size B u and initial bucket fullness B\-Fi is said to contain a bitstream with 
schedule {(f r (n), b(n)) 9 n=0J t ...} if the bucket does not overflow under the following conditions. At *o, d 0 bits are 
inserted into the leaky bucket on top of the B\— F\ bits already in the bucket, and the bucket begins to drain at rate Rl bits 
per second. If the bucket empties, it remains empty until the next insertion. At time t it i > 1, d it bits are inserted into the 
bucket, and the bucket continues to drain at rate R\ bits per second. In other words, for i > 0, the state of the bucket just 
prior to time r, is 

b 0 = B l -F l (C-24) 
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bi+r = max {0, fc, + 4 - /?i('«+r-'<) } - 



(C-25) 



The leaky bucket does not overflow if b t + d t < B\ for all i > 0. 

Equivalently, the leaky bucket contains the bitstream if the graph of the schedule of the bitstream lies between two 
parallel lines with slope R u separated vertically by B } bits, possibly sheared horizontally, such that the upper line begins 
at Fj at time t 0 , as illustrated in Figure C-5. Note from Figure C-5 that the same bitstream is containable in more than 



one leaky bucket Indeed, a bitstream is containable in an infinite number of leaky buckets. 



bits 




bits 




to t { t 2 t 3 



time 



Figure C-5 - Illustration of the leaky bucket concept 



If a bitstream is contained in a leaky bucket with parameters (RtJB^Fi), then when it is input with peak rate /?, to a 
hypothetical reference decoder with parameters R=R U B=B U and F=F it then the HRD buffer does not overflow or 
underflow. 



Minimum buffer size and minimum peak rate 



If a bitstream is contained in two leaky buckets with parameters (R u Bi,Fi) and (R21B2JF2)* then it is also contained in any 
leaky bucket with parameters (RJB,F) where a)R,<R< R 2 , b)B> B^R), and c) F> F mi „(R) and B min (R) and F^R) are 
defined by 



J5 min (/?) = afi fl +(l-a)B n+1 , 



(C-26) 



F^CR) = aF n + (1 - a)F a + u 



(C-27) 



and 



a = (R n+l -R)^(R n+l -R n ). 



(C-28) 



For R<R U 

B m in(R) = i?i + - R)T 



(C-29) 



F min (R) = F iy 



(C-30) 



where T = - to is the duration of the bitstream (i.e., the difference between the decoding times for the first and last 
pictures in the bitstream). And for R > 



(C-31) 



(C-32) 
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Thus, the leaky bucket parameters can be linearly interpolated and extrapolated. 

Alternatively, when the bitstream is communicated to a decoder with buffer size B, it is decodable Drovided > /? rm 

and F> F^Bl where for B n >B>B tt + lf F * m " W 

RminiP) = aJt n + (1 - <x)R n +i (C-33) 

F min {B) = aF n + (1 - a)F rt+I (C-34) 

a = (5 — /?„+,) -s- (Z?„ - 2?„+i). (C-35) 
For£>#i, 

Rmin(B) = R\ — (B — B\) +T (C-36) 



^ min(B) — F\* (C-37) 

For Z? < Zfo, the stream may not be decodable. 

In summary, the bitstream is guaranteed to be decodable in the sense that the HRD buffer does not overflow or 
underflow, provided that the point (R,B) lies on or above the lower convex hull of the set of points (0,5,+/?, 7), (/?,,£,), 
(RnJ3 n ), as illustrated in Figure C-6\ The minimum start-up delay necessary to maintain this guarantee is F m - n (R) -i- R. 




R (bits/sec) 



Figure C-6 - Further illustration of the leaky bucket concept 



An HRD with buffer size B and initial decoder buffer fullness F with peak input rate R shall perform the tests B > B min (R) 
and F> F min (R), as defined above, for any conforming bitstream with LB parameters andThall 
decode the bitstream provided that B > B^iR) and F > F min (R). 

C.2.2.4 Encoder considerations 

The encoder can create a bitstream that is contained by some given N leaky buckets, or it can simply compute N sets of 
leaky bucket parameters after the bitstream is generated, or a combination of these. In the former, the encoder enforces 
the N leaky bucket constraints during rate control. Conventional rate control algorithms enforce only a single leaky 
bucket constraint. A rate control algorithm that simultaneously enforces N leaky bucket constraints can be obtained by 
running a conventional rate control algorithm for each of the N leaky bucket constraints, and using as the current 
quantisation parameter (QP) the maximum of the QP's recommended by the N rate control algorithms. 

Additional sets of leaky bucket parameters can always be computed after the fact (whether rate controlled or not), from 
the bitstream schedule for any given /?„, from the iteration specified in subclause C.2.2.2. 
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Annex D 

Supplemental enhancement inf rmation 

(This annex forms an integral part of this Recomm ndation | International Standard) 

D.l Introduction 

This annex defines supplemental enhancement information that provides data constructs that are synchronous with the 
video data content Each sei_payload( ) defines PayloadType and PayloadSize parameters. This annex defines 
supplemental enhancement information (SEI) that provides a data delivery mechanism construct that is delivered 
synchronous with the video data content SEI assists in the processes related to decoding or display of video. SEI is not 
required for reconstructing the luma or chroma samples by a video decoder, and decoders are not required to process this 
information for conformance to this Recommendation | International Standard. 
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D.2 SEI payload syntax 



cp 5 n»vIo;idf PavlnaHTvrw* Pavln^d^i?!* 1 / 


Category 


Descriptor 


ifV" PavlnadTvnp 1 i 






temnoral rpfprpncpf PavloadSi^p i 


7 




else iff PavloadTvne = = 2 • 1 






clock time^tamnf PavloadSiyp ^ 


7 




else if/ PavloadTvnp — — 






nanscan rertf Pavlnad^i^p • 


7 




pl<;e i fY PflvlnaHTvnp — — A\ 

I'lot' 11^ X CI Jf lUall x y Ut — — *■+ J 






buffering neriodf PavIoadS!i7P i 


7 




else iff PavloadTvne *> ^ 






hrd niptiirpf P^vload^i7p i 


/ 




! else iff PavloadTvnp f\ \ 




. — _ 


unci (JctyiUaUl x ayiUaUOlZC ^ 


/ 




pIqp iff PavIoaHTTvTv* 7 ^ 

tioc 11^ x ayiuau J. V L/C — — / j 






usci udut^icgibierea^iiii 1 10 rayioaooize ^ 


7 




pIqp iff PavlriftdTvrw* Q \ 

CliC ll\ rayiUaU 1 y L/C — 






uscr_aaia_unrcgisierea( r^ayioaaoize j 


7 




eise jrayioaa i ype — ~ zf ) 






random_access_point( PayloadSize ) 


7 




eise ii\ Jrayioaa i ype = = iu ) 






ref_pic_buffer_iiianagernent_repetition( PayloadSize ) 


7 




pica if/ 'Prt\i\r\r»A'T'imo 1 1 \ 

eise ii\ r ayioaa i ype — = 1 1 y 






spare_piciure^ r^ayioauoize ) 


7 




*»lcf» iff PavlnaHTvrvp IT \ 

cjbc ii^ tr dyiuiiu i ype — — iz _/ 




r 


scene liiior indiion\ x ayioaooize ) 


7 




f»lep» iff PjivlnaHTvnp IT \ 






qiiHqati infnrmiitirtnf Po\/lr»i»dQi-7#> "\ 
DUL/odj IJldllVJU^ .T ay lUaUOIiG ^ 


/ 




pIqp iff P^vlnndTvnp id ^ i 

cioc n ^ x ay ivjdu x y Lie — — jl*t y 






snhspn lavpr phflrarfpri^tipcf Pavlnnd^i-zp i 


7 




else iff PavloadTvne = = 1 S ^ 






subseo characteristic^f PavloadSi^p i 


7 




else 






reserved sei messapef Pavload^i^p i 


7 




if( !byte^aligned( ) ) { 






| bit_equal_to_one 


7 


f(D 


while( !byte_aligned( ) ) 






bit_equal_to_zero 


7 


f(D 


} 






} 
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D.2.1 Temp ral reference SEI message syntax 



temporal_reference(Pay 1 oadS i ze ) { 


Category 


Descriptor 


progressive_scan 


7 


u(l) 


bottom_field_flag /* zero if progressive_scan is 1 */ 


7 


u(l) 


six_reserved_one_bits 


7 


f(6) 


temporal_ref_vaIue 


7 


u(v) 


} 
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D.2.2 Clock times tamp SEI messag syntax 



clock_timestamp(PayloadSize ) { 


Category 


j Descriptor | 


progressive_scan 


7 


u(l) J 


bottom_field_flag /* zero if progressive_scan is 1 */ 


7 




six_reserved_one_bits 


7 


f(6) 


counting_type 


7 




full_timestamp_flag 


7 


1 U W \ 


discontinuity_flag 


7 


f "(1) j 


count_dropped 


7 


u(l) [ 


nframes 


7 


j "(8) ! 


if( fu]l_timestamp_flag ) { 






seconds_value /* 0,...,59 */ 


7 


u(6) | 


mimites_value /* 0,...,59 */ 


7 


1 u ( 6 > 1 


hours.value /* 0,. . .,23 */ 


7 


u(5) 


bh_count = 41 






} else { 




J 


seconds_flag 


7 


ufn 1 

1 


bit__count = 25 






if( seconds_flag ) { 







seconds__value /* range 0,...,59 */ 


7 


u \ u / J 


minutes_flag 


7 


* * 


bit_count += 7 






if( minutes_flag ) { 







minu tes_value /* 0,. . . ,59 */ 


7 J 


u(6) ! 


hours_flag 


7 j 


u(l) 1 


bit_count += 7 






if( hours__flag ) { 







hours_value /* 0,. . .,23 */ 


7 [ 


u(5) J 


bit__count += 5 






} 







} 




1 


} 







while( !byte_ahgned( ) ) { 







bit_equaI_to_one 


7 j 


f(D 


bit_count++ 






} 






if( PayioadSize-(bit_count»3) > 0 ) 






time_offset 


7 j 


i(v) 


} 
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D.2.3 Pan-scan rectangle SEI message syntax 



p2ul_SCaH — reci\ rayiuduoiz*? j \ 


Category 


Descriptor 


p a n_scan_ rec t_id 


7 


e(v) 


nan scan rect left offset 


7 


e(v) 


pan_scan_rect_right_offset 


7 


e(v) 


pan_scan_rect_top_offset 


7 


c(v) 


pan_scan__reet_bottom_offset 


7 


e(v) 


} 







D.2.4 Buffering period SEI message syntax 



buffering__period( PayloadSize ) { 


Category 


Mnemonic 


seq_parameter_set_id 


7 


ue(v) 


if( naLhrd^flag s=« 1 ) { 






for( k = 0; k <= pdb_count; k++ ) 






initial_pre_dec_removaLdelay[ k ] 


7 


u(16) 


} 






if( vcljird_flag = = 1 ) { 






for(k = 0; k <= pdb_count; k++ ) 






initial_pre_dec_removal_delay[ k ] 


7 


u(16) 


} 






prev_buf__period_duration 


7 


ue(v) 


} 







D.2.5 HRD picture SEI message syntax 



hrd_picture( PayloadSize ) 


Category 


Descriptor 


pre_dec_removal_deIay 


7 


ue(v) 



D.2.6 Filler payload SEI message syntax 



filler_payload( PayloadSize ) { 


Category 


Descriptor 


for( k as 0; k < PayloadSize; k++) 






fillerjbyte 


7 


f(8) = 0xFF 


} 
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user__data_registered_itu_t_t35( PayloadSize ) { 


Category 


Descriptor 


itu_t_ t35_cou n try_code 


7 


b(8) 


if( country_code != Oxbb ) 






i= 1; 






else { 






itu_t_t35_country_code_extension__byte 


7 


b(8) 


i = 2; 






} 






do { 






itu_t_t35_payload_byte 


7 


K8) 


i++ 






} while( i < PayloadSize ) 






) 







D.2.8 User data unregistered SEI message syntax 



user_data_arbi trary( PayloadSize ) { 


Category 


Descriptor 


i = 0 






do { 






user_data_arbitrary_payload_byte 


7 


b(8) 


i++ 






} while( i < PayloadSize ) 






} 







D.2.9 Random access point SEI message syntax 



random_access_point( PayloadSize ) { 


Category 


Descriptor 


preroll.count 


7 


ue(v) 


postroll_count 


7 


ue(v) 


exact_match_flag 


7 


u(l) 


brokenjinkjlag 


7 


u(l) 


} 







D.2.10 Reference picture buffer management Repetition SEI message syntax 



ref_pic_buffer_management_repetition( PayloadSize ) { 


Category 


Descriptor 


originaLframe_num 


7 


u(v) 


ref_pic_buffer_managemejit( ) 






} 
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D.2.11 Spare picture SEI messag syntax 



spare_picture( Fayioaaoize j i 


Vitlvgvt J 


Descri Dior 


delta_ frame_nuni 


7 


ue(v) 


num_spare_pics_minus 1 


7 




_/ I /\. I ^ »-a »-» »■/-»*» mimic 1 -1.1 * 111 1 J 

tor( i = O; i < num_spare_pics_minusi-ri , i-»-t- y \ 






delta_spare_frame_nu m 


7 




ref_area_indicator 


7 




if( refjarea_indicator = = 1 ) 






for( j = U; j < numoer_or_nfiDS_in_pic, ^ 






ref_mb_Jndicator 


7 




else ir( rei_area__inaicator z ) \ 






MbCnt = u 






do ( 






Tern ran lcn&tli 


7 


ue(v) 


MbCnt = MbCnt + zero_run_length + 1 






) while( MbCnt <= MAX W MB_ADDRESS ) 






} 






} 







r \ 

D.Z12 Scene information SEI message syntax 



scene Jnformation( PayloadSize ) { 


Category 


Descriptor 


scene_id 


7 


u(8) 


scene_transition_type 


7 


ue(v) 


if( scene_transition_type > 3 ) 






second_scene_id 


7 


u(8) 


} 







D.2.13 Sub-sequence information SEI message syntax 



subseq_information( PayloadSize ) { 


Category 


Descriptor 


su bseq_Iay er_nu m 


7 


ue(v) 


subseq_id 


7 


ue(v) 


last_picture_flag 


7 


u(l) i 


if( more_sei_payload_data( ) ) 






stored_frame_cnt 


7 


ue(v) 


} 
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Sub-sequence layer characteristics SEI message syntax 
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subseq_layer_characteristics( PayloadSize ) { 


Category 


Descriptor 


1 do { 






average_bit_rate 


7 


u(16) 


average_frame_rate 


7 


u(16) 


} while( more_sei__payload_data( ) ) 






} . 






Sub-sequence characteristics SEI message syntax 


subseq_characteristics( PayloadSize ) { 


Category 


Df^cri nt r»r 

vvi)l>l 1UIUI 


subseq_Jayer__num 


7 


ue(v) 


subseq_id 


7 


ue(v) 


durationJFlag 


7 


u(D 


if (durationjflag) 






subseq_duration 


7 


u(32) 


average_rate_flag 


7 


u(l) 


i i f ( a verage_rate_flag) { K 






ayerage_bit_rate 


7 


u(16) 


a verage_f rame_ra te 


7 


u(16) 


} 






num_referenced_subseqs 


7 


ue(v) 


for (n = 0; n < num_referenced_subseqs; n++) { 






ref_subseq_Iayer_num 


7 


ue(v) 


ref_subseq_id 


7 


ue(v) 


) 






) 







D.2.16 Reserved SEI message syntax 



reserved_seLmessage( PayloadSize ) { 


Category 


Descriptor 


for( i=0; i<PayloadSize; i++ ) 






reserved_sei_message_payload_byte 


7 


b(8) 


} 







T>3 SEI payload semantics 

D.3,1 Temporal reference SEI message semantics 

progressive_scan: This parameter indicates whether the current picture has progressive or interlaced scan timing. 

bottom_fieId_flag: When progress ive_scan is 0, this parameter indicates whether the temporal reference is for the top 
(0) or bottom (1) field. Shall be 0 if progressive_scan is 1. 

six_reserved_one_bits: Reserved for future backward-compatible use by ITU-T | ISO/IEC. Shall be equal to the binary 
string '111111" unless and until specified otherwise by ITU-T | ISO/IEC. A decoder conforming to this Recommendation 
| International Standard shall ignore the value of these bits. 
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temporal_ref_value: This parameter indicates a number of clock ticks as a multiplier of num_units_jn_tick for the 
current time_scale. It is used for conveying local relative timing information. 

The number of bytes used by temporal_reflvalue shall remain constant for the video stream and shall be equal to 
PayloadSize - 1 bytes. For a temporal_ref_val ue encoded using n bytes, th temporal_reference contains the remainder 
of a clock tick counter modulo 256 n . 

T>3J2 CI ck times tamp SEI message s mantles 

The contents of the clock timestamp SEI message specify a time_offset which indicates the display or capture time 
computed as 

equivalentjimesiamp = ((HH * 60 + MM) * 60 + SS) * time_scale + NF * num_units_in_tick + TO, (D-l) 
in units of ticks of a clock with clock frequency equal to time_scale Hz. 

progressive_scan: This parameter indicates whether the current picture is in progressive or interlaced scan format. 

bottom_field_flag: When progressive_scan is 0, this parameter indicates whether the temporal reference is for the top 
(0) or bottom (1) field. Shall be 0 if progressive_scan is 1. 

six_reserved__one_bits: Reserved for future use by ITU-T | ISO/DEC. Shall be equal to the binary string '111111*. A 
decoder conforming to this Recommendation | International Standard shall ignore the value of these bits. 

counting_type: A 5-bit parameter that specifies the method of dropping values of the nframes parameter as defined in 
Table D-l. 



Table D-l - Definition of counting_type values 



Value (binary) 


— < 

Interpretation 


00000 


no dropping of nframes count values and no use of 
time offset 


00001 


no dropping of nframes count values 


00010 


dropping of individual zero values of nframes count 


00011 


dropping of individual max_pps values of npictures 
count 


00100 


dropping of the two lowest (value 0 and 1) nframes 
counts when seconds^ value is zero and minutes_value is 
not an integer multiple of ten 


00101 


dropping of unspecified individual nframes count values 


00110 


dropping of unspecified numbers of unspecified nframes 
count values 


00111 -11111 


reserved 



fuILtimestamp Jlag indicates whether the nframes parameter is followed by seconds_value or seconds_flag. 

discontinuity _flag indicates whether the time difference between the current value of equivalent_timestamp and the 
valu of equivalent_timestamp computed from the last previously-transmitted clock timestamp can be interpreted as a 
true time difference. A value of 0 indicates that the difference represents a true time difference. 

count_dropped indicates the skipping of a count using the counting method specified by counting_type. 
nframes indicates the value of NF used to compute the equivalent^ mestamp. Shall be less than 

max_fps = Ceil ( time_scale + num_units_in_tick ) (D-2) 

If counting_type is TXXHO* and count_dropped is 1, nframes shall be 1 and the value of nframes for the last previous 
picture in display order shall not be equal to 0 unless discontinuity_flag is equal to 1. 

If counting_type is 00011* and count_dropped is 1, nframes shall be 0 and the value of nframes for the last previous 
picture in display order shall not be equal to max_fps - 1 unless discontinuity_flag is equal to 1. 
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If counting_type is 00100* and count.dropped is 1, nframes shall be 2 and the indicated value of SS shall be zero and the 
indicated value of MM shall not be an integer multiple of ten and nframes for the last previous picture in disDlav orHor 
shall not be equal to 0 or 1 unless discontinuity_flag is equal to 1. 

If counting_type is 00101' or '1 10' and count_dropped is 1, nframes shall not be equal to one plus the value of nframes 
for the last previous picture in display order modulo max.fps unless discontinuity_flag is equal to 1. 

seconds_flag indicates whether seconds_ value is present when fuH_timestamp_flag is 0. 

seconds_value indicates the value of SS used to compute the equivalent_timestamp. Shall not exceed 59 If notpresent 
the last previously-transmitted seconds_value shall be used as SS to compute the equivalent_timestamp. 

minutes_flag indicates whether seconds_value is present when full_timestamp_flag is 0 and seconds_flag is 1. 

minutes value indicates the value of MM used to compute the equivalent_timestamp. Shall not exceed 59 If not 
present, the last previously-transmitted minutes_value shall be used as MM to compute the equivalent.timestamp. 

hours_flag indicates whether seconds_value is present when full_timestamp_flag is 0 and seconds flag is 1 and 
nunutes_flag is 1. - e ,s 1 <mu 

hours.value indicates the value of HH used to compute the equivalent.timestamp. Shall not exceed 23 If not present 
the last previously-transmitted hours_value shall be used as ////to compute the equivalent.timestamp. 
bit_equal_to_one is a single bit which shall be equal to 1. 

tin.e_offset ^jff 68 the valu€ ° { T° to com P ute * e equivalent_timestamp. The number of bytes used to represent 
"™V?r ? Shal, „ be ^ Ual to Pa y' oadS ^ " (bit_count » 3), where bit.count is computed as specified in sXSuse 
D.2.2. If ume_offset ,s not present, the value 0 shall be used as TO to compute the equivalent.timestamp ^ 

D.3.3 ■ Pan-scan rectangle SET message semantics 

The pan-scan rectangle SEI message parameters define the coordinates of a rectangle relative to the cropping rectangle of 
temlTmlli^&T **" coordlnate of ihis * defined in units of 1/16* sample spacing relative to the 

pan_scan_rect_id contains an identifying number which may be used as specified externally to identify the purpose of 
he pan-scan rectangle (for example, to identify the rectangle as the area to be shown on a particular display device or as 
the area that contains a particular actor in the scene). 

pan_scan_rect_left_off S et, P an_scan_rect_right_offset, pan_scan_rect_top offset, and 

pan_scan_rect bottom.offsel t specify, as signed integer quantities in units of 1/16* sample spacing relative to the luma 
sampling grid, the location of the pan-scan rectangle. 

the pan-scan rectangle is defined, in units of 1/16* sample spacing relative to the luma sampling grid, as the area of the 
rectangle with horizontal coordinates from 16 * cropping_rectJeft + pan_scan_rectJeft_offset to 16 * 116 * 
£>ic_width ,n_mbs_minusl + 1) - cropping_rect_right] + pan_scan_rect_right_offset - 1 and with vertical coordinates 
from 16 cropping_rect_top + pan_scan_rect_top_offset to 16 * [16 * (pic^height_in_mbs_minusl + 1) - 
cropp,ng_rect_bottom] + pan_scan_rect_bottom_offset - 1, inclusive. If this rectangular area includes samples outside 
of the cropping rectangle, the region outside of the cropping rectangle may be filled with synthesized content (such as 
black video content or neutral grey video content) for display. 

D.3.4 Buffering period SEI message semantics 

A Buffering Period is defined as the set of pictures between two instances of the Buffering Period SEI message The 
seq_parameter_set_id indicates the sequence parameter set that contains the sequence level HRD attributes. 

seq_parameter_set_id indicates the sequence parameter set that contains the sequence level HRD attributes. 

^ff!H r ?r d r C -y e K mOV ?u de,ay j ?? SyntaX elemem re P resents the de fcy "^ween the time of arrival in the pre-decoder 
buffer of the first bit of the coded data associated with the first picture following the Buffering Period SEI message 
(including all NAL data in the case that the HRD pertains to the NAL) and the time of removal of the coded data 
associated with the picture from the pre-decoder buffer. It is in units of a 90 kHz clock The 
m,t.al_pre_dec_removal_deIay syntax element is used in conjunction with the pre-decoder buffers as specified in Annex 
C. A value of zero is forbidden. 

prev_buf_period_durati n. This syntax element represents the duration of the subset of the video sequence contained 
m the previous Buffering Period. The interpretation of the syntax element is as a number of clock ticks (see Annex D) 
The prev buf per,od_durauon syntax element is used in conjunction with the pre-decoder buffers as specified in Annex 
C. A value of zero is forbidden. 
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D.3.5 HRD picture SET message semantics 

pr dec removaI_delay: This syntax element indicates how many clock ticks (see Annex C) to wait after r moval from 
me HRD pre-decoder buffer of the previous picture before removing from the buffer the picture data immediately 
following the SEI message which contains the element. This value is also used to calculate an earliest possible Ume of 
arrival of picture data into the pre-decoder buffer, as defined in Annex C. 

D.3.6 Fill r payload SEI messag semantics 

This message contains a series of PayloadSize bytes of value OxFF, which can be discarded, 
filler.byte shall be a byte having the value OxFF. 

D.3.7 User data registered by ITU-T Recommendation T.35 SEI message semantics 

This message contains registered user data as specified by ITU-T Recommendation T.35. 

itu_t_t35_country_code shall be a byte having a value specified as a country code by ITU-T Recommendation T.35. 
itu_t_t35_country_code_extension_byte shall be a byte having a value specified as an extended country code by ITU-T 
Recommendation T.35. 

itu_t_t35_payioad_byte shall be a byte containing user data registered as specified by ITU-T Recommendation T.35. 



D .3.8 User data arbitrary SEI message semantics 

This message contains arbitrary user data, the contents of which are not specified by this Recommendation | International 
Standard. 

us r_data_arbitrary_payload_byte shall be a byte having a value not specified by this Recommendation | International 
Standard. 

NOTE - Users of this Recommendation | International Standard should exercise care in the use of the user data arbitrary SEI 
message to avoid the carriage of date content in a form likely to conflict with the data content format of other users (e g., avoiding 
conflict by using a fixed mulu-byte prefix identifier within the payload content). 



D.3.9 Random access point SEI message semantics 

The random access point SEI message indicates the recovery point of decoder output after starting decoding from a 
random access entry point. All decoded pictures at or subsequent to the recovery point in output order are indicated to be 
correct or approximately correct in content. Decoded pictures produced by starting the decoding process at the entry 
point may not be correct in content until the indicated recovery point, and the decoding process starting at the entry point 
and ending at the recovery point may contain references to pictures not available in the multi-picture buffer. 

The entry point is indicated as a pre-roll count relative to the position of the SEI message in units of coded frame 
numbers prior to the frame number of the current picture. The recovery point is indicated as a post-roll count in units of 
coded pictures subsequent to the current picture at the position of the SEI message. 

preroll_count indicates the entry point for the decoding process. Decoding should have started at or prior to the stored 
picture having the frame number equal to the frame number of the next slice minus the preroll_count in modulo 
MAX_FN arithmetic. 

postrolLcount indicates the recovery point of output. All decoded pictures in output order are indicated to be correct or 
approximately correct in content after the stored picture having the frame number equal to the frame number of the next 
slice incremented by postroll_count in modulo MAX_FN arithmetic. 

exact_match_flag indicates whether decoded pictures at and subsequent to the recovery point in output order obtained 
by starting the decoding process at the specified entry point shall be an exact match to the pictures that would be 
produced by a decoder starting at the last prior IDR point in the NAL unit stream. The value 0 indicates that the match 
may not be exact and the value 1 indicates that the match shall be exact. 

If decoding starts from an entry point indicated in a random access point SEI message, all references to unavailable 
stored pictures shall be inferred as references to sample values given by Y=Cb=Cr=128 (mid-level grey) for purposes of 
determining the conformance of the value of exact_match_flag. 

broken_link_flag indicates the presence or absence of a splicing point in the NAL unit stream at th location of the 
random access point SEI message. If broken_link_flag is equal to 1, pictures produced by starting the decoding process 
at the last previous IDR point may contain undesirable visual artifacts due to splicing operations and should not be 
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displayed until the indicated random access recovery point in output order. If broken_link_flag is equal to 0, no 
indication is given regarding any potential presence of visual artifacts. 

If a sub-sequence information SEI message is transmitted in conjunction with a random access point SEI message in 
which broken_link_flag is equal to 1 and if subseqjayer_num is 0, subseqjd should be different from the latest 
subseq_id for subseq Jayer_num equal to 0 that was decoded prior to the entry point. If broken_link_flag is equal to 0, 
the subseq_id in sub-sequence layer 0 should remain unchanged. 

A buffering period SEI message should be transmitted at the location of the random access entry point indicated in the 
random access point SEI message in order to establish initialisation of the HRD buffer model. 

D.3.10 Reference picture buffer management Repetition SEI message semantics 

The Reference picture buffer management repetition SEI message is used to repeat memory management control 
operation commands that were located earlier in decoding order. 

origina!_frame_num identifies the picture were the repeated memory management control operation originally 
occurred. 

ref_pic_buffer_managament( ) shall contain a copy of the reference picture buffer management syntax elements of the 
picture whose frame_num was original_frame_num. 

D.3.1 1 Spare picture SEI message semantics 

The spare picture SEI message indicates that certain macroblocks, called spare decoded macroblocks, in one or more 
decoded stored pictures resemble the co-located macroblocks in a certain decoded picture, called the target picture, so 
much that any of these spare decoded macroblocks can be used to replace a co-located incorrect decoded macroblock in 
the target picture in the multi-frame buffer and in decoder output. Decoded pictures that contain spare macroblocks are 
called spare pictures. 

The picture that contains the next slice or data partition in decoding order is herein referred to as the current picture. The 
frame_num of the current picture is herein denoted as CurrFrameNum. 

d Ita_frame_num identifies the target picture whose spare pictures and macroblocks are specified later in the message. 
Let TargetFrameNum be the frame_num of the target picture, and the target picture is the stored picture having the 
TargetFrameNum. TargetFrameNum is calculated as follows 

TargetFrameNum = CurrFrameNum - delta_frame_num 
if( TargetFrameNum < 0 ) 

TargetFrameNum = MAX_FN + TargetFrameNum (D-3) 

num_spare_pics_minusl specifies the number of pictures which contain spare picture or macroblocks for the target 
picture. 

delta_spare_frame_num specifies to which spare picture the following spare picture information in the current loop 
count belongs. For the first spare picture of the message, CandidateSpareFrameNum is equal to TargetFrameNum - 1 if 
TargetFrameNum is greater than 0 and MAX.FN - 1 otherwise. For later spare pictures, CandidateSpareFrameNum is 
the SpareFrameNum of the previous loop round minus 1 if SpareFrameNum is greater than 0 and MAX_FN - 1 
otherwise. For each loop round, SpareFrameNum is calculated as follows: 

SpareFrameNum = CandidateSpareFrameNum - delta_spare_frame_num 
if( SpareFrameNum < 0 ) 

SpareFrameNum = MAXJFN + SpareFrameNum (D-4) 

ref_area_indicator specifies how the locations of spare macroblocks are coded. ref_area_indicator 0 indicates that all 
macroblocks of the spare picture are spare macroblocks.. ref_area_indicator 1 indicates an uncompressed spare 
macroblock map. ref_area_indicator 2 indicates a compressed spare macroblock map. A spare macroblock map consists 
of flags for each macroblock location of a picture. A flag shall be 0 if the macroblock location in the spare picture is a 
spare macroblock and 1 otherwise. 

If ref_area_indicator is 1, there is a reflmbjndicator for each macroblock address of the spare macroblock map in 
raster scan order, reflmbjndicator 0 indicates that the macroblock is a spare macroblock, and ref_mb_indicator 1 
indicates that the macroblock is not a spare macroblock. 

If ref_area_indicator is 2, a spare macroblock map between a spare picture and the target picture is compressed. The 
coded macroblock map for loop_count equal to 0 is the spare macroblock map between the target picture and the first 
spare picture. A coded macroblock map for loop_count greater than 0 is generated by applying an exclusive or operation 
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between the previous spare macroblock map and the current spare macroblock map. The coded macroblock map is 
scanned in counter-clockwise box-out order as specified in subclause 8.3.4.1. The number of consecutive zeros in the 
scanning order is indicated in zero_run Jength. 

D3.12 Scene information SEI message semantics 

A scene is herein defined as a set of pictures in decoding order captured with one camera. The scene information SEI 
message is used to label scenes with identifiers. The message concerns the next slice or data partition in decoding order. 

scene_id: Pictures in a scene shall share the same value of scene_jd. Consecutive scenes in decoding order should not 
have the same value of scene_id. If the next slice or data partition in decoding order belongs to a picture that includes 
contents from two scenes, scene_id is the scene identifier of the former scene in decoding order. 

The following values of scene_transition_type are valid: 



Table D-2 - Scene transition types. 



Value 


Description 


0 


No transition 


1 


Fade-out 


2 


Fade-in 


3 


Unspecified transition from or to constant color 


4 


Dissolve | 


5 


Wipe 


6 


Unspecified mixture of two scenes 


Other values 


Reserved 



If scene_transition_type is greater than 3, the next slice or data partition in decoding order belongs to a picture that 
includes contents from two scenes. 

second_scene_id is present if the next slice or data partition in decoding order belongs to a picture that includes contents 
from two scenes. second_scene_id is the scene identifier of the latter scene in decoding order. 

D.3.13 Sub-sequence information SEI message semantics 

The sub-sequence information SEI message is used to indicate the position of a picture in data dependency hierarchy that 
consists of sub-sequence layers and sub-sequences. 

A sub-sequence layer contains a subset of the coded pictures in a coded data stream. Sub-sequence layers are numbered 
with non-negative integers. A layer having a larger layer number is a higher layer than a layer having a smaller layer 
number. The layers are ordered hierarchically based on their dependency on each other so that a layer does not depend on 
any higher layer and may depend on lower layers. In other words, layer 0 is independently decodable, pictures in layer 1 
may be predicted from layer 0, pictures in layer 2 may be predicted from layers 0 and 1, etc. The subjective quality 
increases along with the number of decoded layers. 

A sub-sequence is a set of coded pictures within a sub-sequence layer. A picture shall reside in one sub-sequence layer 
and in one sub-sequence only. A sub-sequence shall not depend on any other sub-sequence in the same or in a higher 
sub-sequence layer. A sub-sequence in layer 0 can be decoded independendy of any other sub-sequences and previous 
long-term reference pictures. 

The sub-sequence information SEI message concerns the next slice or data partition in decoding order. The picture which 
the next slice or data partition belongs to is herein referred to as the target picture. 

subseq Jayer_num indicates the sub-sequence layer number of the target picture. 

subseq_id identifies the sub-sequence within a layer. Consecutive sub-sequences within a particular layer in decoding 
order shall have a different subseq_id from each other. 

last_picture_flag equal to 1 signals that the target picture is the last picture of the sub-sequence (in decoding order). 

stored_frame_cnt is 0 for the first stored picture of the sub-sequence. For each coded frame belonging to the sub- 
sequence in decoding order, stored_frame_cnt shall be incremented by I, in modulo MAX_FN operation, relative to the 
previous stored frame that belongs to the sub-sequence. 

D3.14 Sub-sequence layer characteristics SEI message semantics 

The sub-sequence layer characteristics SEI message indicates the characteristics of sub-sequence layers. 
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A pair of average bit rate and average frame rate characterizes each sub-sequence layer. The first pair of average bit rate 
and average frame rate signals the characteristics of sub-sequence layer 0. Hie second pair, if present, signals the 
characteristics of sub-sequence layers 0 and 1 jointly. Each pair in decoding order signals the characteristics for a range 
of sub-sequence layers from layer number 0 to the layer number that is incremented by one from the previous upper limit 
of layer numbers. The values are in effect from the point they are decoded until an update of the values is decoded. 

average_bit_rate gives the average bit rate in units of 1000 bits per second. All NAL units in the range of sub-sequence 
layers specified above are taken into account in the calculation. The average bit rate is calculated according to the 
decoding time of the NAL units. Value zero means an undefined bit rate. 

average_frame_rate gives the average frame rate in frames/(256 seconds) of the sub-sequence layer. Value zero 
indicates an undefined frame rate. 

D3.15 Sub-sequence characteristics SEI message semantics 

The sub-sequence characteristics SEI message indicates the characteristics of a sub-sequence. It also indicates inter 
prediction dependencies between sub-sequences. 

This message applies to the next sub-sequence in decoding order having the indicated subseq_layer_num and 
subseq_id. This sub-sequence is herein called the target sub-sequence. 

duration_flag equal to zero indicates that the duration of the target sub-sequence is not specified. 

subseq_duration indicates the duration of the target sub-sequence in clock ticks of a 90-kHz clock. 

average_rate_flag equal to zero indicates that the average bit rate and the average frame rate of the target sub-sequence 
are unspecified. 

averagejbit^rate gives the average bit rate in (1000 bits)/second of the target sub-sequence. A!l NAL units of the target 
sub-sequence are taken into account in the calculation. The average bit rate is calculated according to the decoding time 
of the NAL units. 

average_frame_rate gives the average frame rate in frames/(256 seconds) of the current sub-sequence. 

num_referenced_subseqs gives the number of sub-sequences which contain pictures mat are used as reference pictures 
for inter prediction in the pictures of the target sub-sequence. 

ref_subseq_Iayer_num and ref_subseq_id identify a sub-sequence that contains pictures that are used as reference 
pictures for inter prediction in the pictures of the target sub-sequence. 

D.3.16 Reserved SEI message semantics 

This message consists of data reserved for future backward-compatible use by ITU-T | ISO/EEC. Encoders conforming 
to this Recommendation | International Standard shall not send reserved SEI messages until and unless the use of such 
messages has been specified by ITU-T | ISO/IEC. Decoders conforming to this Recommendation | International 
Standard that encounter reserved SEI messages shall discard their content without effect on the decoding process, except 
as specified in future Recommendations | International Standards defined by ITU-T | ISO/IEC. 

reserved_sei_message_payload_byte is a byte reserved for future use by ITU-T | ISO/IEC. 
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Annex E 
Video usability information 

(This annex forms an integral part of this Recommendation | International Standard) 

E.l Introduction 

This Annex specifies those parts of the sequence parameter set and the picture parameter set that are not required for 
determining the decoded values of samples. The parameters specified in this annex can be used to facilitate the use of 
the decoded pictures or facilitate the resource allocation of a decoder by restricting certain video parameters beyond 
those limits specified by Annex A. Decoders are not required to process VUI sequence parameters for conformance to 
this Recommendation | International Standard. 

For each of the parameters of this Annex, default values are defined in the semantics subclause. The syntax includes 
flags that allow avoiding the signalling of groups of parameters. If a specific group of parameters is not coded, the 
default values for the parameters become effective. 
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E.2 VUI syntax 

E.2.1 VUI sequence parameters syntax 



vui_seq_parameters( ) { 


Category 


Descriptor 1 


aspect_ratio_info_flag 


0 


u(D 1 


if( aspect_ratio_jnfo_flag ) { 






aspect_ratio_info 


0 


b(8) | 


if( aspect_ratio_info = = "Extended SAR" ) { 




1 


sar_width 


0 


u(8) 


$ar_height 


0 


u(8) | 


1 




! 


} 






video_signal_type_flag 


0 


u(l) 


it( video_signal_type_flag ) { 






video_format 


0 


"(3) 


video_range_flag 


0 


"(1) 


colour_descriptionJflag 


0 


u(l) 


if( colour_description_flag ) { 




I 


colour_primaries 


0 


b(8) 


transfer_characteristics 


0 


b(8) 


matrix_coefficients 


0 


b(8) j 


} 




j 


} 




1 


chroma_location_flag 


0 


u(l) 


if (chromajocation_flag) { 




j 


chroma_location_frame 


0 


e(v) J 


chroma Jocationjield 


0 


e(v) | 


} 






timing_information_flag 






if( timing_information_flag ) { 




! 


num_units_in_tick 


0 


u(32) 


time_scale 


0 


u(32) 


fixed_frame_rate_JFIag 

~ : ■ — — 


0 


u(D | 


J 




1 


nal_nra_nag 


0 




ir( nal_nra_llag == 1) 






hrd_parameters( ) 






vcl_hrd_flag 


0 


u(l) 








h rd_par a meters ( ) 




1 


it( ( nal_hrd Jlag = = 1 1 1 ( vcl_hrd_flag = = 1 ) ) { 






low delay hrd 






removal_time_toIerance 


0 


ue(v) | 


} 






bitstream_restriction_flag 


0 


u(D 


if( bitstream_restriction_flag ) { 


0 


u(D 


motion_vectors_over_pic_boundaries_flag 


0 j 


u(l) 


minimum_compression_per_pic__reversed 


0 


e(v) 1 
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minimum_compressioii_per_macroblock„reversed 


0 


e(v) 


log2_maxiinu m_mv_iength_vertical 


0 


e(v) 


log2_maximum_mv_iengui_noiizont:ai 


o 




i 






} 







E.2.2 HRD parameters syntax 



hrd _parameters( ) { /* coded picture buffer parameters */ 






pdb cnt 


0 


ue(v) 


bit rate scale 


0 


u(4) 


coded pic buffer size scale 


0 


u(4) 


for( k=l; k<=pdb cnt; k++ ) { 






bit rate valuefk] 


0 


ue(v) 


coded pic buffer size value[ k ] 


0 


ue(v) 


vbr cbr flag[ k ] 


0 


u(l) 


1 






} 







E.23 VUI picture parameters syntax 

i V 



vui_pic_parameters( ) { 


Category 


Descriptor 


frame_cropping_flag 


I ! 


ue(v) 


if( frame_cropping_flag ) { 






frame_cropping_rect_Ieft_offset 


1 


ue(v) 


frame_cropping_rect_right_offset 


1 


ue(v) 


frame_cropping_rect_top_offset 


1 


ue(v) 


I frame_cropping_rect_bottom_offset 


1 


ue(v) 


) 






} 







E3 VUI semantics 

E3.1 VUI sequence parameters semantics 

aspect_ratio_info_flag: A flag that, when 1, signals the presence of the aspect_ratio_info. If the flag is 0, then the 
following default values shall apply: aspect_rati<fjnfo = 0. 

aspect_ratio_info is an eight-bit integer which defines the value of sample aspect ratio. Table E-i shows the meaning of 
the code. If aspect_.ratio Jnfo indicates Extended S AR, sampIe_aspect_ratio is represented by sar_ width and sar_height. 
The sar_width and sarjieight shall be relatively prime. If aspect_ratio_info is zero or if either sar_width or sar_height 
are zero, the sample aspect ratio shall be considered unspecified or specified externally. 



Table E-l - Meaning of sample aspect ratio 



aspect_ratio_info 


Sample aspect ratio 


0000 0000 


Undefined or specified externally 


0000 0001 


1:1 ("Square") j 


0000 0010 


12:11 (625-type for 4:3 picture) 


0000 0011 


10:1 1 (525-type for 4:3 picture) 


0000 0100 


16:1 1 (625-type stretched for 16:9 picture) 


0000 0101 


40:33 (525-type stretched for 16:9 picture) 
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oooo 0110 


24*1 1 fHalf-wiHf* 4«'*t fnr 


0000 0111 


20*1 1 (Half-u/iHf* A-1 €nr 


0000 1000 


32:1 1 (Half-wide ifvQ for 


0000 1001 


80*33 CHalf-wiHe 1 A-Q 


0000 1010 


io.i i ^j-wiuc 't.j lor o^3j 


0000 1011 


15:1 1 C2/3-wide'4-^ fnr 


1 0000 1 100 


24*1 1 f2/3-wiHf» 1rvQ fnr ! 
» • * * — ' wiuc iu.7 ior uzj j 


0000 1101 


20*1 1 f2/3-widp ifvQ for *;o<*\ 


0000 1110 


16*1 1 r3/4-widi» 4-^ for /V}<\\ 


0000 1111 


40*33 f3/4-wiHf* 4*^ for 


0001 0000 


64:33 (3/4- wide 16:9 for 625) 


0001 0001 


160:99 (3/4-wide 16:9 for 525) 


0001 OOlOto 1111 1110 


Reserved 


1111 1111 


Extended SAR 



forbidd^ ^ 8 " bk UnSignCd ime8er WhiCh indiCateS h0ri20nta, size of asP^t ratio. A zero value is 

sar_height is an 8-bit unsigned integer which indicates the vertical size of sample aspect ratio. A zero value is forbidden. 
video_signal type flag: A flag that, when 1, signals the presence of video signal information If 
25ri£j ,S °* Allowing default values shall apply: video.format = 8 ',0r, viaeo_rl„ g e = 0, 

video format: This is a three bit integer indicating the representation of the pictures before being coded in accordance 
w.th th,s Recommendat,on | International Standard. Its meaning is defined in Table E-2. If the video signal tS i no 
present in the bitstream then the video format may be assumed to be "Unspecified video format" °- S ' gnaI - type( } ,s not 



Table E-2 - Meaning of video_format 



video_f©rmat 


Meaning 


000 


Component 


001 


PAL 


010 


NTSC 


011 


SEC AM 


100 


MAC 


101 


Unspecified video format 


110 


Reserved 


111 


Reserved 



video -range Jlag indicates the nominal black level and range of the luminance and chrominance signals as derived from 
fc Yf E PB, and E pr analogue component signals as follows: 
If video_range_flag=0: 

Y = round(219*E'Y+ 16) 
Cb = round( 224 * E'pb + 128 ) 
Cr = round( 224 * E>r + 128 ) 

If video_range_flag= 1 : 

Y = round( 255 * E'y ) 

Cb = round( 255 * E'pb + 128 ) 
Cr = round( 255 * E'pr + 128 ) 

If video_signal_type_flag is zero, video__range shall be inferred to have value 0 (a nominal range of Y from 16 to 235) 
SSSS£ZSSil m ' " ■'' *• • °' — and 
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colour_primaries: This 8-bit integer describes the chromaticity coordinates of the source primaries, and is defined in 
Table E-3. 



Table E-3 - Colour Primaries 



Value 


Primaries 


0 


Reserved 


1 


ITU-R Recommendation BT.709 




primary 


X 


y 




green 


0,300 


0,600 




blue 


0,150 


0,060 




red 


0,640 


0,330 




white D65 


0,3127 


0,3290 


2 


Unspecified video 






Image characteristics are unknown. j 


3 


Reserved 


4 


ITU-R Recommendation BT.470-2 System M 




primary 


X 


y 




green 


0,21 


0,71 




blue 


0,14 


0,08 




red 


0,67 


0,33 




white C 


0,310 


0,316 


5 


ITTJ-R Recommendation BT.470-2 System B, G 




primary 


X 


y 




green 


0,29 


0,60 




blue 


0,15 


0,06 




red 


0,64 


0,33 




white D65 


0,3127 


0,3290 


6 


SMPTE 170M 








primary 


X 


y 




green 


0,310 


0,595 




blue 


0,155 


0,070 




red 


0,630 


0,340 




white D65 


0,3127 


0,3290 


7 


SMPTE 240M (1987) 






primary 


X 


y 




green 


0,310 


0,595 




blue 


0,155 


0,070 




red 


0,630 


0,340 




white JD65 


0,3127 


0,3290 


8 


Generic film ( colour filters using Illuminant C ) 




primary 


X 


y 




green 


0,243 


0,692 ( Wratten 58 ) 




blue 


0,145 


0,049 ( Wratten 47 ) 




red 


0,681 


0,319 (Wratten 25) 


9-255 


Reserved 



If video_signal_type_flag is zero or colour_description is zero, the chromaticity is unspecified or specified externally, 
transfer^characteristics: This 8-bit integer describes the opto-electronic transfer characteristic of the source picture, 
and is defined in Table E-4. 
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Table E-4 - Transf r Characteristics 



Value 


| Transfer Characteristic 


0 


Reserved 


I 1 


ITU-R Recommendation BT.709 
V= 1, 099 L c °> 4 5. 00 99 

| for 1 >L C £ 0,018 
V = 4,500 L c j 

I for 0,018 >L r >0 


1 


I Unspecified video j 
Image characteristics are unknown. 


1 o 

1 3 


Reserved 


4 


1 1 U-R Recommendation BT.470-2 System M 
Assumed display gamma 2,2 


5 


1 ITU-R Recommendation BT.470-2 System B, G ] 
Assumed display gamma 2,8 


1 a 


SMPTE 170M 

V= 1,099 L c °» 4 5- 0,099 

for 1 >L C > 0,018 
V = 4,500 L c 

! for 0,018 >L^>0 


7 


SlClPTE240M(1987) 
V=1,1115L C 0,45. 01115 

for L c > 0,0228 
V = 4,0L C 

for 0,0228> L c 


8 \ 


Linear transfer characteristics j 
i.e.V = L r J 


9 


Logarithmic transfer characteristic (100:1 range ) 
V= 1.0-log I0 (Lc)/2 

for 1=L C = 0.01 

v = o.o 

for 0.01> L c 


10 


Logarithmic transfer characteristic ( 3 1 6.22777: 1 range ) f 
for 1 =L C = 0.0031622777 

V = 0.0 

- for 0.003 1622777 >L n \ 


11-255 ] 


Reserved j 



If video_signal_type_flag is zero or colour.description is 
specified externally. 

matrix__coefficients: This 8-bit integer describes the matrix 
signals from the green, blue, and red primaries, as specified in 

Using this table: 



zero, the transfer characteristics are unspecified or are 

coefficients used in deriving luminance and chrominance 
Table E-5. 



E' y is analogue with values between 0 and I 

E 'R> E'G> and E*b are analogue with values between 0 and 1 

E'pb and E'pr are analogue between the values -0,5 and 0,5 

White is defined as E'r = E'g = E'b = 1 

White equivalently given by E'y = 1, E'pb = 0, E'pr = 0 

E'Y = ***E* R + ( 1 E'g + *s*E'b 



.1100641A_I_> 



ITU-T Rec H.264 (2002 E) 



193 



ISO/IEC FCD 14496-10 

E'pb = 05{ E*R - E'Y >K1-K R ) 
E'pb = 0.5 ( E'b - E'Y HO -Kb) 

Table E-5 - Matrix Coefficients 



Value 


Matrix 


0 


Reserved 


1 


nU-R Recommendation BT/709 
Jfcs 0,7152; Kjt = 0,2126 


2 


Unspecified video 

Image characteristics are unknown. 


3 


Reserved 


4 


FCC 

Xb = 0,59; =0,30 


5 


nU-R Recommendation BT.470-2 System B, G: 
K c = 0,587; = 0,299 


6 


SMPTE 170M 

K G = 0,587; X* = 0,299 


7 


SMPTE 240M (1987) 
K c = 0,701; = 0,212 


8-255 


Reserved ~" 



If video_signaUype_flag is zero or colour_description is zero, the matrix coefficients are assumed to be undefined 
specified externally. 

chroma Jocation_flag: A flag that, when 1, signals the presence of the chroma location information. If the flag is 
then the following default values shall apply: chroma Jocation Jrame = 0, chroma Jocation_field = 0. 
chroma JocationJFrame specifies the 4:2:0 sampling structure according to Table E-6 and Figure E-l. 



Table E-6 - Chroma Sampling Structure Frame 



Value 


Sampling Structure 


0 


undefined 


1 


Frame according to Figure E-l Chroma Sample Mode 1 


2 


Frame according to Figure E-l Chroma Sample Mode 2 


3 


Frame according to Figure E- 1 Chroma Sample Mode 3 
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X Lunr, a Sample 

O Chroma Sample Mode 1 

A Chroma Sample Mode 2 

Chroma Sample Mode 3. 
co-located with Luma 



Figure E-l - Luma and chroma sample types 

chroma Jocation_field specifies the 4:2:0 sampling structure according to table E-7 and Figure E-2. 

Table E-7 - Chroma Sampling Structure Frame 



Value 


Sampling Structure 


0 


undefined 


1 


Frame according to Figure E-2 Chroma Sample Mode 1 


2 


Frame according to Figure E-2 Chroma Sample Mode 2 


3 


Frame according to Figure E-2 Chroma Sample Mode 3 ~] 



— X— EH — X — K 
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X— X— X X X 

M—x— m x la- 

OA OA O 

^ < X X X X 

X -ESI — X— E3- 
OA OA O 
X— X— X x x 



Top Held 

-XXX — 


Bottom Field 


o o 
-A- ^ — 




-s s 


— X X X 




* Hr— 

o O 


X X X 


— 2£ 


o o 
-fr it — 




-E3 — 


X X X 




+ 

o o 




)0Z ^ — 



X Luma Sample 

O Chroma Sample Mode 1 

Chroma Sample Mode 2 in top field 

)S£ Chroma Sample Mode 2 in bottom field, 
co-located with Luma 

Chroma Sample Mode 3 in top field, 
co-located with Luma 

Chroma Sample Mode 3 in bottom field 



Figure E-2 - Luma and chroma association 
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timing_inforniation_fla g : A flag that, when 1, signals the presence of time unit information If 
Uming_information_flag is set to 0, then the following default values shall apply: num_units_in_tick = 0, time_scale = 0, 
fixed_frame_rate = 0. 

num_units_in_tick is the number of time units of a clock operating at the frequency time_scale Hz that corresponds to 
one increment of a clock tick counter. A clock tick is the minimum interval of time that can be represented in the coded 
data. For example, if the clock frequency of a video signal is (30 000) * 1001 Hz, time_scale may be 30 000 and 
num_units_in_tick may be 1001 . If num_units_in_tick is 0, the duration of the clock tick is unspecified. 

time scale is the number of time units which pass in one second. For example, a time coordinate system that measures 
lime"using a 27 MHz clock has a time.scale of 27 000 000. If time.scale is 0, the duration of the clock tick specified 
above is unspecified.' 

fixed frame_rate_flag: A bit that, if equal to 1, indicates that the temporal distance between the HRD output times of 
any two consecutive frames or fields in output order as defined in Annex C is a constant. If equal to 0, the temporal 
distances between HRD output times of consecutive frames or fields in output order as defined in Annex C may not be 
constant. 

nal _hrd_flag If nal_hrd_flag = M \ the multiplexed NAL and VCL stream complies with a hypothetical reference 
decoder (HRD) as specified in Annex C. In this case, the HRD parameters follow the nal_hrd_flag in the sequence 
parameter set syntax. If nal_hrd_flag = '0', the multiplexed NAL and VCL stream is not guaranteed to comply with an 
HRD. No default values are specified. 

NOTE - If nal_hrd_flag = 0 the maximum buffer sizes and bit rates specified in Annex A apply. 
vcl_hrd_flag- If vcl_hrd_flag = T, the VCL bitstream complies with a hypothetical reference decoder (HRD) as 
specified in Annex C. In this case, the HRD parameters follow the vcl_hrd_flag in the sequence parameter set syntax. If 
vcl_hrd_flag = '0', the VCL bitstream is not guaranteed to comply with an HRD. 

NOTE - If vcl_hrd_flag = 0 the maximum buffer sizes and bit rates specified in Annex A apply, 
r moval _time_to!erance: This syntax element indicates the number of clock ticks (see Annex C) of deviation allowed 
between the pre-decoder buffer removal times (see subclauses C.2.5 and C.3.5) and the accumulated Buffering Period 
capture time (see subclauses C.2.4 and C.3.4). It is encoded as a universal VLC, with all values allowed. A value of *0' 
implies that, at each measurement point (i.e. each picture preceding a Buffering Period SEI message), the removal time 
shall exactly match the capture time. 

bitstream_restriction_flag: A flag that, when 1, signals the presence of bitstream restriction information. If 
bitstream_restriction_flag is set to 0, then the following default values shall apply: 
motion_vectors_over_pic_boundaries_flag = 1, minimum_compression_per_pic_reversed = 4, 
minimum_compression_per_macroblock_reversed = 1, log2_maximum_mv_length_vertical = 16, 
log2_maximum_mv_length_horizontal =16. 

motion_vectors_over_pk_boundaries_flag equal to 0 indicates that no motion vector refers to samples outside the 
picture boundaries. motion_vectors_over_pic_boundaries_flag equal to 1 indicates that motion vectors may refer to 
samples outside the picture boundaries. 

minimum_compression_per_pic_reversed and minimum_compressioii_per_macroblock_reversed advise the 
decoder about the minimum compression ratio (corresponding to a maximum coded picture or macroblock size 
respectively). A value of n for either of the two indicates a minimum compression ratio of l:n. Annex A defines the 
numbering range for both values. 

log2_maximum_mv_length_vcrtical and log2jinaximum_mvJength_horizontal indicate the maximum value of the 
absolute of a non-predicted vertical or horizontal motion vector component, in units of either V4 or 1/8 sample, depending 
on the value of motion_vector_resolution. A value of n asserts that no absolute value of a motion vector component is 
bigger than 2**n V* pel or 1/8* pel units. Note: the high default value is restricted in Annex A for some profile/level 
combinations. Furthermore, the maximum vector length is restricted by the picture size. 

E.3.2 HRD parameters semantics 

pdb_cnt: This syntax element indicates the number of pre-decoder buffers (PBDs) in the HRD. A value of pdb.cnt 
equal to '0' is not allowed. 

bit_rate_scale: Together with bit_rate_value{k], this syntax element defines the maximum input bit rate of the k-th PDB 
in an HRD. 

bit_rat _valne(k]: Together with bit_rate_scale, this syntax element defines the maximum input bit rate of the k-th PDB 
in an HRD. The actual bit rat in bits per second is given by: 

bit_rate[ k ] = bit_rate_value[k] * 2 t6+ «*~e-c. (E-l) 
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SX£i5™K2?D S t0£ether With Coded -P ic - buff --^--Ie[k] to define the maximum input bit 



Sfhf W^n^-^H^'S 1 iS use , d J°S e,her With coded_pic_buffer_size_vaIue to define the pre-decoder buffex size 
of the k-th PDB in an HRD. The actual buffer size in bits is given by 

coded_pic_buffer_size[k] = coded_pic_buffer_size_value r k] * 2 (4 + coded -J' ic - bu «^-sae-scaie) ^ ^ 

/VRpI >r - f !. a8 eqU , al t ?„' 0 '*. th i? Synt3X eIement indicates ^ ,he Pre-decoder buffer operates in variable bit rate 
(VBR) mode. If equal to '1', it indicates constant bit rate (CBR) operation. variaoie oit rate 

^7? p e,a iRn rd:If ! OW - d f ay -^ Ifow delay hrdisequalto 

1 the HRD operates ,n low-delay mode. In low-delay mode, only one HRD buffer may be selected and \ b£ Sres 
wh,ch violate the HRD removal time rules at the pre-decoder buffer are permitted. It is expected thaTsSh b e p etudes 
occur only occasionally, but not mandatory. olg P ,CIUres 

E.3.3 VUI picture parameters semantics 

S ra Z- C ^ n n L - fla8 1, 1 Siga f $ the P resence of bitstream restriction information. If frame croppi„g_flag is set to 
0, then the following default values shall apply frame_crop P ing_rec t _Ieft = 0, frame cropping. refe S - 0 
frame_cropping_rect_top = 0,frame_cropping_rect_bottom = 0. -cropping_rect_nght - 0, 

frame cropping^rectjeft, frame_cro PP ing_rect_right, frame_cropping_rect top, frame cropping rect bottom 
define the area of the luma picture internal array which shall be the output oHhe decoding proce^Th XSd?2£ 
of these offsets consist of non-negative integer values, and the output of the decoding process is deLed as the a^ea 

S/" H t . reCta " 8 i e C ° n,aining ,Uma ?amp,eS With horizontal coordinates from croppinTreS left to 
16*(pic_w,dth_.n mbs minusl + 1 )-( C ropping_rect_right + 1 ) and with vertical coordinates from c Sntrec too 
to 16*( pic_he.ght_,n_mbs_minusl + 1 )-( cropping_rect_bottom + 1 ), inclusive cropping_rect_top 
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